Commit Graph
13568 Commits
Author SHA1 Message Date
John Colvin de9d47cd00 Include subscription id in ds websocket updates 2024-10-30 19:31:39 +00:00
John Colvin 461f690463 fix: Define BoooleanConstant before StringConstant
Otherwise the former masks the latter and prevents it from matching
2024-10-30 19:31:39 +00:00
John Colvin a425d3e6ba Log event data in websocket open, close, error callbacks 2024-10-30 19:31:39 +00:00
John Colvin fe81b3107f refactor: Bring back Material DataGrid
Fix the column resize bugs that made it previously unusable
2024-10-30 19:31:39 +00:00
John Colvin d5d12c56c9 refactor: Improve react webapp state management
* Add queryTextResetTime as part of the CannedQuery state
  (the QueryControl compares the canned query click time with the time of the
  last user input, and applies the text from the most recent change)

* Replace 'isRunning' (bool) with a 'queryState' (string) to reflect the subscription status
  value received from the server. Less indirection this way.

* Replace inappropriate useMemo call w/ useEffect
2024-10-30 19:31:39 +00:00
John Colvin 8979d5def8 cleanup: Get rid of magic const (subscriptionId) in webapp App.jsx 2024-10-30 19:31:39 +00:00
John Colvin 922cf3170b build: Copy configuration profiles to publish dir 2024-10-30 19:31:39 +00:00
John Colvin 5da807b5f0 cleanup: Remove unnecessary deps from csprojs 2024-10-30 19:31:39 +00:00
John Colvin c3507ab77b build: Change publish_data_stream.py to publish the sln
We were publishing individual projects as a workaround for the
dependency issue (bug) that caused some projects to incorrectly target
net8.0. The fact that publishing individual projects "just worked" was
probably dumb luck. Now that the underlying issue has been addressed, we
can resume publishing the sln as a whole.
2024-10-30 19:31:39 +00:00
John Colvin 521346a5c5 dev: Install docker in devcontainer
So we can start various services like sparkd, pubsub, ds publishers, ds
webapp, etc. from inside the devcontainer.
2024-10-30 19:31:39 +00:00
John Colvin a94d1f6f67 dev: Remove dotnet8 from devcontainer
We'll upgrade to dotnet8 eventually, but now isn't the time (too much
more important work to be done).
2024-10-30 19:31:39 +00:00
John Colvin c69adca7ba dev: Run devcontainer on host network
So we don't need to worry about publishing the right ports (and don't
need to rebuild the container everytime the list of ports changes).
2024-10-30 19:31:39 +00:00
John Colvin 212bd432af fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2024-10-30 19:31:39 +00:00
John Colvin 4361ac6519 Add HttpsRedirection middleware first in the webapp pipeline
Middleware handlers are invoked in the order in which they are added. Any middleware
that is added before HttpsRedirection may can prevent the request from being redirected.
As of now, only development-specific middleware is added ahead of HttpsRedircetion.
2024-10-30 19:31:09 +00:00
John Colvin 93dc4c9c1f build: Copy configuration profiles to build dir
So they are readily available to publish/deploy scripts.
2024-10-30 19:31:09 +00:00
John Colvin ebc39f203d Fix benign typo in Scanner/Parser msbuild targets
The output path referenced a non-existant msbuild metadata prop "RelativeOutputDir"
(the actual prop is called "RelativeDir"). Also, the OutputFile path is relative to the
project dir (the 'MSBuildProjectDir' prop), not the file being included.
2024-10-30 19:31:09 +00:00
John Colvin ec88ffdbf3 fix: Catch exception when creating TcpClient 2024-10-30 19:31:09 +00:00
John Colvin 26fecc73c6 fix: Catch WebSocketException in DataStreamSubscription 2024-10-30 19:31:09 +00:00
John Colvin 05715eb489 fix: Add null check to NonIncGroupByStream 2024-10-30 19:31:09 +00:00
John Colvin 70019c59e5 fix: Add null check to TableAccumulator2 2024-10-30 19:31:09 +00:00
John Colvin 66f9e9aa4c fix: Add self-signed certificate to Dockerfile for ds webapp 2024-10-30 19:31:09 +00:00
John Colvin 6caa98a6b3 fix: Make SystemSubscriptionMapPublisher threadsafe
The PubSubState.onTagMapChanged event may not be invoked on
the same thread as doWork.
2024-10-30 19:31:09 +00:00
John Colvin 40930628fb fix: Add null check in TimeBasedFilter.cs 2024-10-30 19:31:09 +00:00
John Colvin d439e3e695 Reduce task wait/timeout code in AsyncTCPConnector 2024-10-30 19:31:09 +00:00
John Colvin aae50cd5f8 Add a few canned queries to AwsProd.json for ds webapp 2024-10-30 19:31:09 +00:00
John Colvin 587e340071 fix: Copy canned queries to clipboard on click
Instead of trying to update the text directly, which causes the front end
to overwrite the user's changes everytime the data updates from then on.
2024-10-30 19:31:09 +00:00
John Colvin 156ec8f3a8 fix: Convert invalid doubles to nan in ds webapp serializer 2024-10-30 19:31:09 +00:00
John Colvin 7ca96f128b Increase datagrid maxheight in ds webapp to 1000px 2024-10-30 19:31:09 +00:00
John Colvin 9aec630100 Improve error display in ds webapp 2024-10-30 19:31:09 +00:00
John Colvin 831cf840a7 fix: Dispose existing sub on subscribe req in ds webapp
Even if new sub has parse error. This fixes a bug where the previous
subscription would continue firing and updating the frontend, when it
should just display the error info to the user and cease updates.
2024-10-30 19:31:09 +00:00
John Colvin 8940325cfd fix: Get rid of "UseSpa" calls in webapp Production environment
This was leftover from a misguided attempt to fix a bug that turned out
to be due to an omitted environment var:
ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=Microsoft.AspNetCore.SpaProxy
2024-10-30 19:31:09 +00:00
John Colvin aaf75acd89 Define webapp specific env vars in generic datastream app deploy script
For now, this makes things easier, as I don't need to write a special script just for the webapp
2024-10-30 19:31:09 +00:00
John Colvin 13200de777 Add config profile and Dockerfile args to deploy script 2024-10-30 19:31:09 +00:00
John Colvin 26f8a2a4ed Copy DataStream WebApp 'dist' to 'wwwroot' in deploy script 2024-10-30 19:31:09 +00:00
John Colvin 5b77e4a36d Tweak DataGrid style in DataStream WebApp 2024-10-30 19:31:09 +00:00
John Colvin 13668374b7 Add AwsProd config for DataStream WebApp 2024-10-30 19:31:09 +00:00
John Colvin 405c6889c4 fix: Add VITE_ prefix to imported env vars
For the DataStream WebApp client
2024-10-30 19:31:09 +00:00
John Colvin b2a15cc33b Fix urls for DataStreamPublisher AwsProd.json 2024-10-30 19:31:09 +00:00
John Colvin e31d1ad69b Add benzinga news streams to AwsProd.json
This is a config profile for DataStreamPublisher, designed to be deployed
to aws (on public internet).
2024-10-30 19:31:09 +00:00
John Colvin c88d1b26f0 fix build warnings: Make props nullable strings in SharedStreamConfig 2024-10-30 19:31:09 +00:00
John Colvin b1446993b4 whitespace: Format PubSubServer.json (indentation issues) 2024-10-30 19:31:09 +00:00
John Colvin db0d415b5a Add a few tasks to run DataStream WebApp 2024-10-30 19:31:09 +00:00
John Colvin 3f1ad91beb fix: Import websocket url from environment var 2024-10-30 19:31:09 +00:00
John Colvin 639d17a625 Add scripts to deploy data stream apps 2024-10-30 19:31:09 +00:00
John Colvin 072b8b2b02 Add SpaServer to Production environment
For Bts.DataStreamWebApp.Server
2024-10-30 19:31:09 +00:00
John Colvin 658ffcad27 Change DirectoryBrowser requestPath to "/browse"
So we can view a psuedo file explorer for the webapp by requesting the "/browse" path
(only activated in Development environment).
2024-10-30 19:31:09 +00:00
John Colvin 2b0f6e52b2 Publish Bts.DataStreamWebApp.Server along with PubSub & DataStream 2024-10-30 19:31:09 +00:00
John Colvin d721bba4db cleanup: Remove unused code in Bts.DataStreamWebApp.Client 2024-10-30 19:31:08 +00:00
John Colvin c387eb81c9 Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2024-10-30 19:31:08 +00:00
John Colvin 707ebcfbc4 Hardcode webSocketUrl to aws server in DataStreamWebApp client 2024-10-30 19:31:08 +00:00