Commit Graph
6129 Commits
Author SHA1 Message Date
John Colvin 2bde71f09b Add api server to docker compose 2024-09-22 18:47:05 +00:00
John Colvin bdc439df62 Change api server ports so they don't conflict w/ ds webapp 2024-09-22 18:47:05 +00:00
John Colvin 1f85b124fb Don't log "initializing spark config" if it has already been initialized 2024-09-22 18:46:10 +00:00
John Colvin d9b2db778f Use dependency injection in spark api server 2024-09-22 18:46:10 +00:00
John Colvin 2acd911e39 Add SparkLoggerAdapter for Msft ILogger 2024-09-22 18:46:10 +00:00
John Colvin 2cf72277b3 fix: Ensure spark config init and login are only performed once
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.

It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2024-09-22 18:36:36 +00:00
John Colvin ce8ae14a54 fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2024-09-22 18:36:36 +00:00
John Colvin 45ae320324 Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2024-09-22 18:33:42 +00:00
John Colvin c5e851d0aa Improve error reporting in DataStreamWebSocketServer 2024-09-22 18:33:42 +00:00
John Colvin 79673b7a01 fix build: Fix Spark build after making config init async 2024-09-22 18:33:42 +00:00
John Colvin 2cf7826e4d fix Change Configuration.initialize to be async 2024-09-22 18:33:42 +00:00
John Colvin 8bb67dfba2 refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2024-09-22 18:33:42 +00:00
John Colvin f922215cae refactor: Change Configuration.initialize to be async 2024-09-22 18:33:42 +00:00
John Colvin 8194a07a8e SparkApiService check pt 2024-09-22 18:33:42 +00:00
John Colvin eff979fb2b improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-09-19 20:03:31 +00:00
John Colvin 2dfa44c1e4 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-09-19 20:02:09 +00:00
John Colvin 022cad3122 Add spark service modules designed for Dependency Injection 2024-09-19 20:02:09 +00:00
John Colvin c941fa78d2 fix build (api server): Change namespace to bts.websocket_core 2024-09-19 20:02:09 +00:00
John Colvin 97bc186110 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-09-19 20:02:09 +00:00
John Colvin b2e3a6bf82 Add JsonRpcRequestObservable, AsyncEnumerable 2024-09-19 20:02:09 +00:00
John Colvin f87aa6c4b6 Add json facilities for DataStream websocket api 2024-09-19 20:02:09 +00:00
John Colvin 40a46dd6d7 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-09-19 20:02:09 +00:00
John Colvin 75d04b2cd2 feat: Add JsonRpc types for websocket paradigm 2024-09-19 20:02:09 +00:00
John Colvin de439579cf fix: Fix improper 'OfType' linq cast
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2024-09-19 20:02:09 +00:00
John Colvin 13646a5a16 refactor: Add WebsocketClient class
Basically an ISubject (I/O) wrapper for IWebsocketConnectionGateway
2024-09-19 20:02:09 +00:00
John Colvin 9a1e51eacf refactor: Change WebsocketServerConfig.port to uint32
It should always be a number between 1-65535
2024-09-19 20:02:09 +00:00
John Colvin 9982bd2203 refactor: Rename WebsocketServerConfig proto pkg 'bts.websocket_core'
This matches the namespace used in spark_websocket csproj. Eventually,
I'd like to use conventional capitalization (TitleCase namespaces/types,
etc.), but let's shelf that until we have enough time/energy to do a
more comprehensive refactoring.
2024-09-19 20:02:09 +00:00
John Colvin e7d7352941 refactor: Remove nullability constraint in GetValueOrDefault 2024-09-19 20:02:09 +00:00
John Colvin d2efe037f9 refactor: Use events to avoid circular deps for ws sync/gateway 2024-09-19 20:02:09 +00:00
John Colvin 67e6147633 refactor: Let caller specify nullability of T in json parse fn 2024-09-19 20:02:09 +00:00
John Colvin 37e5003cf9 Merge remote-tracking branch 'origin/development.ps_linux' into development 2024-09-19 05:40:59 +00:00
Taha 33a8e06a3a Fixing implied vols for SPX (market data was getting divide dby the fwd) 2024-09-17 22:17:31 -04:00
John Colvin 56d207c3f1 fix: Preserve vs exp sec type (Option) when computing implied vol
Because the exp keys in the data map have sec type Option.
2024-09-16 13:46:53 -04:00
John Colvin 9172e752e3 ui: Highlight launcher buttons when selected
Namely, the expandable menu buttons in Analytics shell module
2024-09-16 13:46:53 -04:00
John Colvin adc9f2acfc Replace trivial refs to ThreadedBlackboard w/ IBus 2024-09-15 23:45:15 -04:00
John Colvin 41d17e67d2 fix: Move accounts subscription to dispatcher thread
We update a collection that is bound to the view in the callback (mutation
ops throw if performed on a different thread).
2024-09-15 23:45:15 -04:00
John Colvin 5cacff073b fix: Check 'HasTimestamp' before accessing prop in man fill row
Pending adds (i.e., unpersisted new rows) don't have timestamps.
In such cases, the 'Timestamp' getter throws an exception.
2024-09-15 23:45:15 -04:00
John Colvin c8da92c250 Cache/index options for faster lookups in SystemContractData 2024-09-15 23:45:15 -04:00
John Colvin c4e76076e3 fix: Include venue w/ symbol when publishing interp greeks 2024-09-15 23:45:14 -04:00
John Colvin 6a715b9a68 Fix typo in comment 2024-09-15 23:45:11 -04:00
John Colvin 1f28940c95 fix: SelectorList.RemoveAt doesn't need 'convert_back' delegate
So don't throw an exception if it is null.
2024-09-15 23:45:11 -04:00
John Colvin ebff40b88f fix: Don't assume venue is in spread def string
Pass it as an argument to the parse fn. This should be fine for now, but
obviously a singular venue param won't work for cross exchange spreads.
2024-09-15 23:45:11 -04:00
John Colvin a6d14a3ac3 improvement: Move system contracts sub off of dispatcher thread
Also add 200ms throttle. Contracts shouldn't change that often, so we don't need
lightening speed here. Furthermore, such changes may trigger expensive cpu ops.

Note: this change also replaces refs to ThreadedBlackboard w/ IBus (the
former is somewhat tethered to the dispatcher thread, particularly by
its timer_callback).
2024-09-15 23:45:11 -04:00
John Colvin af96bcb9e3 fix: Pass symbol/venue from vs usage cfg to AnalyticsService proto 2024-09-15 23:28:59 -04:00
John Colvin 619702cb04 cleanup: Remove unused code 2024-09-15 23:28:59 -04:00
John Colvin 5baec19aff refactor: Index vol and theos data by symbol/venue tuple
Rather than just symbol, to avoid conflicts when the same symbol exists
at multiple exchanges
2024-09-13 10:50:23 -04:00
John Colvin 39ef30c1e3 cleanup: Remove redundant type VolSurfaceSymbolStructure
There is an identical type (in terms of prop types and corresponding proto indices)
called VolSurfaceSymbolRelationship that is used in more places.
2024-09-13 10:50:23 -04:00
John Colvin c2f2d28914 Move nested vm classes into own files
VolSurfaceSettingsEditor has too much code (still). It is hard to
navigate.
2024-09-13 10:50:23 -04:00
John Colvin e06bc3bdf5 Simplify driver symbol structure in vs settings vm
The code was confusing and difficult to follow.
2024-09-13 10:50:23 -04:00
John Colvin e19c19c013 fix: Raise prop changed event when is_selected changes 2024-09-13 10:50:23 -04:00