Commit Graph
6139 Commits
Author SHA1 Message Date
John Colvin 64bd5eef4f Load IDataStremFactory impls with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of types
through reflection.
2024-11-26 18:53:32 +00:00
John Colvin eb3fad64b0 fix build: Upgrade Newtonsoft.Json 13.0.2 -> 13.0.3 in spark_websocket
The DataStreamSuite.sln build was broken because other projects in the sln were
already using the newer version (can't mix and match).
2024-11-26 18:53:32 +00:00
John Colvin bdfb00aa22 Add c# Variant class, modeled after std::variant in c++ 2024-11-26 18:53:32 +00:00
John Colvin 73d670c0dc fix: Add null check 2024-11-26 18:53:32 +00:00
John Colvin 551430e256 fix: Update source paths for api server docker 2024-11-26 18:53:32 +00:00
John Colvin c57312fb4d feat: Add stream subscriptions to websocket api 2024-11-26 18:53:32 +00:00
John Colvin e0ad6f72c0 fix build (c#): Update source file paths, namespaces, etc.
The c# builds have been broken ever since we reorganized the c++ projects with
stuttering dirs.
2024-11-26 18:53:32 +00:00
John Colvin d947882b0e refactor: Break shared mem md code up into smaller files
For easier reading/navigation
2024-11-26 16:32:09 +00:00
John Colvin b523ab22f0 refactor: Move domain types into new lib_domain
Also rename files in snake_case (standard naming convention for protos)
https://protobuf.dev/programming-guides/style/
2024-11-25 20:58:11 +00:00
John Colvin 0248b73bde Add api server to docker compose 2024-11-25 15:26:59 +00:00
John Colvin d267cbbb73 Change api server ports so they don't conflict w/ ds webapp 2024-11-25 15:26:59 +00:00
John Colvin f5a638bf8f Don't log "initializing spark config" if it has already been initialized 2024-11-25 15:26:59 +00:00
John Colvin 1b9ce5f9b7 Use dependency injection in spark api server 2024-11-25 15:26:59 +00:00
John Colvin 7fb411adce Add SparkLoggerAdapter for Msft ILogger 2024-11-25 15:26:59 +00:00
John Colvin 40c09084bd fix: Ensure spark config init and login are only performed once
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.

It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2024-11-25 15:26:58 +00:00
John Colvin 5dfd2d96b5 fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2024-11-25 15:26:58 +00:00
John Colvin 94591eee6d Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2024-11-25 15:26:58 +00:00
John Colvin d485621d21 Improve error reporting in DataStreamWebSocketServer 2024-11-25 15:26:57 +00:00
John Colvin de15b0ea4b fix build: Fix Spark build after making config init async 2024-11-25 15:26:57 +00:00
John Colvin bd41e6cb52 refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2024-11-25 15:26:57 +00:00
John Colvin f07b1f2e5f refactor: Change Configuration.initialize to be async 2024-11-25 15:26:57 +00:00
John Colvin c3cd8cf543 SparkApiService check pt 2024-11-25 15:26:57 +00:00
John Colvin cb43af0dd4 improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-11-25 15:26:57 +00:00
John Colvin 9cc0108a01 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-11-25 15:26:57 +00:00
John Colvin 6c1d1a43f5 Add spark service modules designed for Dependency Injection 2024-11-25 15:26:57 +00:00
John Colvin b35b9170e4 fix build (api server): Change namespace to bts.websocket_core 2024-11-25 15:26:57 +00:00
John Colvin 7ebafd8246 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-11-25 15:26:57 +00:00
John Colvin 1fa1dfa01e Add JsonRpcRequestObservable, AsyncEnumerable 2024-11-25 15:26:56 +00:00
John Colvin ec28e12a6f Add json facilities for DataStream websocket api 2024-11-25 15:26:56 +00:00
John Colvin 84b7e74bb6 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-11-25 15:26:56 +00:00
John Colvin 64ffd3050f feat: Add JsonRpc types for websocket paradigm 2024-11-25 15:26:56 +00:00
John Colvin 60de12277b fix: Fix improper 'OfType' linq cast
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2024-11-25 15:26:56 +00:00
John Colvin 00ce3cdf37 refactor: Add WebsocketClient class
Basically an ISubject (I/O) wrapper for IWebsocketConnectionGateway
2024-11-25 15:26:56 +00:00
John Colvin fbdb4f112d refactor: Change WebsocketServerConfig.port to uint32
It should always be a number between 1-65535
2024-11-25 15:26:56 +00:00
John Colvin 2f9f20de06 refactor: Rename WebsocketServerConfig proto pkg 'bts.websocket_core'
This matches the namespace used in spark_websocket csproj. Eventually,
I'd like to use conventional capitalization (TitleCase namespaces/types,
etc.), but let's shelf that until we have enough time/energy to do a
more comprehensive refactoring.
2024-11-25 15:26:56 +00:00
John Colvin c7b5b5df54 refactor: Remove nullability constraint in GetValueOrDefault 2024-11-25 15:26:56 +00:00
John Colvin 9342613501 refactor: Use events to avoid circular deps for ws sync/gateway 2024-11-25 15:26:56 +00:00
John Colvin eb8c97a1b7 fix: Detect integer ticks of all sizes (not just 1)
Also don't divide by zero!

Bugs (fixed):
  * Spark crashes when opening a ladder with integer tick size of 5.
    This was first observed with TAIFEX futures.

    Cause: The ladder throws a DivideByZero exception when calculating
    the number of ticks per dollar. It isn't caught until it bubbles up
    to the scope in which 'Dispatcher.Run' was called. The catch handler
    then deliberately assert crashes.
2024-11-12 16:15:47 -05:00
John Colvin 5517c58724 fix (ui): Display trade time w/ microsecond precision
Was only showing milliseconds before
2024-11-12 11:42:23 -05:00
John Colvin e9bcdd0c65 fix build: Update Newtonsoft.Json to 13.0.3 in Spark.csproj 2024-11-12 10:35:51 -05:00
John Colvin 37e5003cf9 Merge remote-tracking branch 'origin/development.ps_linux' into development 2024-09-19 05:40:59 +00:00
Taha 33a8e06a3a Fixing implied vols for SPX (market data was getting divide dby the fwd) 2024-09-17 22:17:31 -04:00
John Colvin 56d207c3f1 fix: Preserve vs exp sec type (Option) when computing implied vol
Because the exp keys in the data map have sec type Option.
2024-09-16 13:46:53 -04:00
John Colvin 9172e752e3 ui: Highlight launcher buttons when selected
Namely, the expandable menu buttons in Analytics shell module
2024-09-16 13:46:53 -04:00
John Colvin adc9f2acfc Replace trivial refs to ThreadedBlackboard w/ IBus 2024-09-15 23:45:15 -04:00
John Colvin 41d17e67d2 fix: Move accounts subscription to dispatcher thread
We update a collection that is bound to the view in the callback (mutation
ops throw if performed on a different thread).
2024-09-15 23:45:15 -04:00
John Colvin 5cacff073b fix: Check 'HasTimestamp' before accessing prop in man fill row
Pending adds (i.e., unpersisted new rows) don't have timestamps.
In such cases, the 'Timestamp' getter throws an exception.
2024-09-15 23:45:15 -04:00
John Colvin c8da92c250 Cache/index options for faster lookups in SystemContractData 2024-09-15 23:45:15 -04:00
John Colvin c4e76076e3 fix: Include venue w/ symbol when publishing interp greeks 2024-09-15 23:45:14 -04:00
John Colvin 6a715b9a68 Fix typo in comment 2024-09-15 23:45:11 -04:00