Commit Graph
6074 Commits
Author SHA1 Message Date
John Colvin 9798edec08 Add api server to docker compose 2024-08-01 14:03:21 +00:00
John Colvin 27b8d36e1c Change api server ports so they don't conflict w/ ds webapp 2024-08-01 13:35:00 +00:00
John Colvin 65aeac3c73 Don't log "initializing spark config" if it has already been initialized 2024-07-31 08:32:04 -04:00
John Colvin e775bc19b8 Use dependency injection in spark api server 2024-07-31 00:02:11 -04:00
John Colvin a89c4a5e1e Add SparkLoggerAdapter for Msft ILogger 2024-07-31 00:00:29 -04:00
John Colvin 638d46469f fix: Ensure spark config init and login are only performed once
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.

It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2024-07-29 03:13:04 +00:00
John Colvin 00e71ea9bd fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2024-07-28 18:10:55 +00:00
John Colvin b9b49e628a Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2024-07-17 18:01:45 +00:00
John Colvin ba410f194c Improve error reporting in DataStreamWebSocketServer 2024-07-16 11:45:24 -04:00
John Colvin 5940f55be9 fix build: Fix Spark build after making config init async 2024-07-16 11:43:53 -04:00
John Colvin 84534cf469 fix Change Configuration.initialize to be async 2024-07-15 15:03:18 -04:00
John Colvin ca95964ef8 refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2024-07-15 15:00:21 -04:00
John Colvin 2fa46d9f46 refactor: Change Configuration.initialize to be async 2024-07-15 14:59:23 -04:00
John Colvin 44110e43aa SparkApiService check pt 2024-07-14 13:17:29 -04:00
John Colvin 0ba85cdf3d improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-07-12 18:28:47 -04:00
John Colvin 3095253aa9 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-07-12 18:28:46 -04:00
John Colvin 8227304aab Add spark service modules designed for Dependency Injection 2024-07-12 18:28:46 -04:00
John Colvin f09d6badaf fix build (api server): Change namespace to bts.websocket_core 2024-07-09 15:14:29 -04:00
John Colvin e1526e25a7 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-07-09 15:14:26 -04:00
John Colvin 6f623f9a37 Add JsonRpcRequestObservable, AsyncEnumerable 2024-07-09 15:03:30 -04:00
John Colvin 827189cdb4 Add json facilities for DataStream websocket api 2024-07-09 15:03:30 -04:00
John Colvin 123f12ab88 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-07-09 14:59:06 -04:00
John Colvin f51ba8fb56 feat: Add JsonRpc types for websocket paradigm 2024-07-09 14:59:06 -04:00
John Colvin f92964114e fix: Fix improper 'OfType' linq cast
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2024-07-09 14:59:06 -04:00
John Colvin 7104bd787a refactor: Add WebsocketClient class
Basically an ISubject (I/O) wrapper for IWebsocketConnectionGateway
2024-07-09 14:59:06 -04:00
John Colvin 5db92d1f7d refactor: Change WebsocketServerConfig.port to uint32
It should always be a number between 1-65535
2024-07-09 14:59:06 -04:00
John Colvin eb0fcf0b84 refactor: Rename WebsocketServerConfig proto pkg 'bts.websocket_core'
This matches the namespace used in spark_websocket csproj. Eventually,
I'd like to use conventional capitalization (TitleCase namespaces/types,
etc.), but let's shelf that until we have enough time/energy to do a
more comprehensive refactoring.
2024-07-09 14:59:06 -04:00
John Colvin a52b187313 refactor: Remove nullability constraint in GetValueOrDefault 2024-07-09 14:59:06 -04:00
John Colvin 669b059eb2 refactor: Use events to avoid circular deps for ws sync/gateway 2024-07-09 14:59:06 -04:00
John Colvin 36ab8492d3 refactor: Let caller specify nullability of T in json parse fn 2024-07-09 14:48:59 -04:00
John Colvin 4a2735839e Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin 672149dd89 Add more descriptive logs (via stdout) during initialization
The C++ configure/initialize routines were particularly opaque
2024-06-06 09:15:16 -05:00
John Colvin f8fc2a1d96 cleanup: Remove static service Locator class
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
2024-06-06 09:15:16 -05:00
John Colvin 9a7b3cb77c refactor: Reduce code (simplify) in BTS.init
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
2024-06-06 09:15:16 -05:00
John Colvin 8d22e91c0d fix build: Remove references to HighPerformanceTimer
An old custom timer class from Edge that we no longer need. We're
replacing it w/ a dotnet builtin timer class called 'Stopwatch'.
2024-06-06 09:15:15 -05:00
John Colvin 40681c6093 Add MaybeNullWhen(false) attribute to TryGetFirst result
So callers can assume non-null when it returns true (for IEnumerables of
non-null ref types)
2024-06-06 09:15:15 -05:00
John Colvin 44b46033e3 cleanup: Remove HighPerformanceTimer 2024-06-03 13:46:51 +00:00
John Colvin 63306173b1 refactor: Reorganize NumericComparer, support all types
Get rid of redundant UniversalNumericComparer while we're at it
2024-05-25 15:37:33 +00:00
Mohammad Arham 1e64811d93 changed error code 2024-04-03 16:54:51 -04:00
John Colvin 8ae940bcbc fix: Change "init" call to "initAsync" in spark_api_server main
WebsocketDataSync.init was refactored to initAsync
2024-04-01 10:09:23 -04:00
Mohammad Arham 966ccb5f2c fixed order entry 2024-04-01 07:48:36 -04:00
Mohammad Arham 7e3b1de70f bug fix 2024-03-29 00:38:33 -04:00
Mohammad Arham 2a07e3cf3c custom orders working on api 2024-03-29 00:37:04 -04:00
Mohammad Arham 4c013a8886 reorganise code 2024-03-26 18:07:01 -04:00
Mohammad Arham 9509818978 set theos 2024-03-26 18:06:07 -04:00
Mohammad Arham 4fdf3b8b60 just cleaner code 2024-03-26 02:10:06 -04:00
Mohammad Arham ee9a58486d cleaner code same logic 2024-03-25 02:52:51 -04:00
Mohammad Arham fb1697b571 spelling fix 2024-03-24 23:17:02 -04:00
Mohammad Arham 4080d606d8 fixed get trades 2024-03-24 23:16:38 -04:00
Mohammad Arham 7655fd47dd fixed spelling 2024-03-24 23:14:11 -04:00