Commit Graph
14321 Commits
Author SHA1 Message Date
John Colvin 77ffd8145b Include subscription id in ds websocket updates 2025-07-23 16:12:57 -04:00
John Colvin c6308db55f fix: Define BoooleanConstant before StringConstant
Otherwise the former masks the latter and prevents it from matching
2025-07-23 16:12:57 -04:00
John Colvin b7aac2e431 Log event data in websocket open, close, error callbacks 2025-07-23 16:12:57 -04:00
John Colvin e72ef3b775 refactor: Bring back Material DataGrid
Fix the column resize bugs that made it previously unusable
2025-07-23 16:12:57 -04:00
John Colvin 9ec0610be4 refactor: Improve react webapp state management
* Add queryTextResetTime as part of the CannedQuery state
  (the QueryControl compares the canned query click time with the time of the
  last user input, and applies the text from the most recent change)

* Replace 'isRunning' (bool) with a 'queryState' (string) to reflect the subscription status
  value received from the server. Less indirection this way.

* Replace inappropriate useMemo call w/ useEffect
2025-07-23 16:12:57 -04:00
John Colvin be502be538 cleanup: Get rid of magic const (subscriptionId) in webapp App.jsx 2025-07-23 16:12:57 -04:00
John Colvin af819af516 build: Copy configuration profiles to publish dir 2025-07-23 16:12:57 -04:00
John Colvin 69da3d46ab cleanup: Remove unnecessary deps from csprojs 2025-07-23 16:12:57 -04:00
John Colvin 64b4eec999 build: Change publish_data_stream.py to publish the sln
We were publishing individual projects as a workaround for the
dependency issue (bug) that caused some projects to incorrectly target
net8.0. The fact that publishing individual projects "just worked" was
probably dumb luck. Now that the underlying issue has been addressed, we
can resume publishing the sln as a whole.
2025-07-23 16:11:50 -04:00
John Colvin 563610c2cf fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2025-07-23 16:09:22 -04:00
John Colvin c46e694ccd Add HttpsRedirection middleware first in the webapp pipeline
Middleware handlers are invoked in the order in which they are added. Any middleware
that is added before HttpsRedirection may can prevent the request from being redirected.
As of now, only development-specific middleware is added ahead of HttpsRedircetion.
2025-07-23 16:09:22 -04:00
John Colvin b6ee5335e5 build: Copy configuration profiles to build dir
So they are readily available to publish/deploy scripts.
2025-07-23 16:09:22 -04:00
John Colvin 4272f6dc41 Fix benign typo in Scanner/Parser msbuild targets
The output path referenced a non-existant msbuild metadata prop "RelativeOutputDir"
(the actual prop is called "RelativeDir"). Also, the OutputFile path is relative to the
project dir (the 'MSBuildProjectDir' prop), not the file being included.
2025-07-23 16:09:22 -04:00
John Colvin f4e4b56b3e fix: Catch exception when creating TcpClient 2025-07-23 16:08:14 -04:00
John Colvin 542df5ff82 fix: Catch WebSocketException in DataStreamSubscription 2025-07-23 16:08:14 -04:00
John Colvin 997a97be40 fix: Add null check to NonIncGroupByStream 2025-07-23 16:08:14 -04:00
John Colvin 025eb4a4fc fix: Add null check to TableAccumulator2 2025-07-23 16:08:14 -04:00
John Colvin 844c842357 fix: Add self-signed certificate to Dockerfile for ds webapp 2025-07-23 16:08:14 -04:00
John Colvin a7d873f42f fix: Make SystemSubscriptionMapPublisher threadsafe
The PubSubState.onTagMapChanged event may not be invoked on
the same thread as doWork.
2025-07-23 16:08:14 -04:00
John Colvin c8970408a4 fix: Add null check in TimeBasedFilter.cs 2025-07-23 16:08:14 -04:00
John Colvin 3cef34179e Reduce task wait/timeout code in AsyncTCPConnector 2025-07-23 16:08:14 -04:00
John Colvin 349a058567 Add a few canned queries to AwsProd.json for ds webapp 2025-07-23 16:08:14 -04:00
John Colvin 32599bdeff fix: Copy canned queries to clipboard on click
Instead of trying to update the text directly, which causes the front end
to overwrite the user's changes everytime the data updates from then on.
2025-07-23 16:08:14 -04:00
John Colvin 89845ace1c fix: Convert invalid doubles to nan in ds webapp serializer 2025-07-23 16:07:35 -04:00
John Colvin d77a7b6b56 Increase datagrid maxheight in ds webapp to 1000px 2025-07-23 16:07:15 -04:00
John Colvin 17f84a9d77 Improve error display in ds webapp 2025-07-23 16:07:04 -04:00
John Colvin c7ac783570 fix: Dispose existing sub on subscribe req in ds webapp
Even if new sub has parse error. This fixes a bug where the previous
subscription would continue firing and updating the frontend, when it
should just display the error info to the user and cease updates.
2025-07-23 16:06:51 -04:00
John Colvin e296b83a2e fix: Get rid of "UseSpa" calls in webapp Production environment
This was leftover from a misguided attempt to fix a bug that turned out
to be due to an omitted environment var:
ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=Microsoft.AspNetCore.SpaProxy
2025-07-23 16:06:38 -04:00
John Colvin 1474c84848 Define webapp specific env vars in generic datastream app deploy script
For now, this makes things easier, as I don't need to write a special script just for the webapp
2025-07-23 16:06:25 -04:00
John Colvin 97f4268686 Add config profile and Dockerfile args to deploy script 2025-07-23 16:06:11 -04:00
John Colvin 678fb0ac4e Copy DataStream WebApp 'dist' to 'wwwroot' in deploy script 2025-07-23 16:06:00 -04:00
John Colvin 2aa895f339 Tweak DataGrid style in DataStream WebApp 2025-07-23 16:05:46 -04:00
John Colvin 9a56634fc2 Add AwsProd config for DataStream WebApp 2025-07-23 16:05:34 -04:00
John Colvin 1cabc793d8 fix: Add VITE_ prefix to imported env vars
For the DataStream WebApp client
2025-07-23 16:05:18 -04:00
John Colvin c88d0b626b Fix urls for DataStreamPublisher AwsProd.json 2025-07-23 16:04:58 -04:00
John Colvin 757314f110 Add benzinga news streams to AwsProd.json
This is a config profile for DataStreamPublisher, designed to be deployed
to aws (on public internet).
2025-07-23 16:04:36 -04:00
John Colvin f77e1dddc3 fix build warnings: Make props nullable strings in SharedStreamConfig 2025-07-23 16:04:25 -04:00
John Colvin dbe35d4783 whitespace: Format PubSubServer.json (indentation issues) 2025-07-23 16:04:13 -04:00
John Colvin eef090b77e Add a few tasks to run DataStream WebApp 2025-07-23 16:03:50 -04:00
John Colvin 22cab0b4b8 fix: Import websocket url from environment var 2025-07-23 16:03:40 -04:00
John Colvin 9e554c1950 Add scripts to deploy data stream apps 2025-07-23 16:03:08 -04:00
John Colvin 0b32e1d3c9 Add SpaServer to Production environment
For Bts.DataStreamWebApp.Server
2025-07-23 16:02:16 -04:00
John Colvin 0d72746bea Change DirectoryBrowser requestPath to "/browse"
So we can view a psuedo file explorer for the webapp by requesting the "/browse" path
(only activated in Development environment).
2025-07-23 16:01:51 -04:00
John Colvin dffc08c790 Publish Bts.DataStreamWebApp.Server along with PubSub & DataStream 2025-07-23 16:01:19 -04:00
John Colvin 6aea1ddbc2 cleanup: Remove unused code in Bts.DataStreamWebApp.Client 2025-07-23 16:01:05 -04:00
John Colvin da963b8da3 Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2025-07-23 16:00:50 -04:00
John Colvin facd2f109f Hardcode webSocketUrl to aws server in DataStreamWebApp client 2025-07-23 16:00:40 -04:00
John Colvin ff21f3056f Assert that System publishers 'doWork' is always called on same thread 2025-07-23 16:00:19 -04:00
John Colvin 7d1a16a6a6 fix: Make InternalPublisherQueue thread safe 2025-07-23 16:00:05 -04:00
John Colvin debaaa0d04 Improve error reporting in DataStreamWebSocketServer 2025-07-23 15:59:52 -04:00