Compare commits

...
Author SHA1 Message Date
John Colvin ddcc85bbc0 webapp check pt? 2024-09-18 14:02:40 +00:00
John Colvin 1b9c4fba2f refactor: Move logging into its own lib
Also moved a few logging deps from lib_spark -> system:
* bts_chrono.h
* bts_no_copy.h
* defer.h
* dispatcher.h/cpp
* file_utils.h/cpp
* type_name.h
2024-09-18 14:02:40 +00:00
John Colvin 49c7683d58 refactor: Move system_config.proto to lib_system 2024-09-18 14:02:40 +00:00
John Colvin a8a474dd7f refactor: Move thread_manager to lib_system 2024-09-18 14:02:40 +00:00
John Colvin 19450f4029 cleanup: Remove unnecessary includes 2024-09-18 14:02:40 +00:00
John Colvin 86d54fdd68 refactor: Move threads.proto to lib_system 2024-09-18 14:02:40 +00:00
John Colvin b413f519b5 refactor: Move warn_ctrl.h into lib_system
This is another logging dep
2024-09-18 14:02:40 +00:00
John Colvin 9f3e0898b9 build: Link boost_filesystem in lib_system 2024-09-18 14:02:40 +00:00
John Colvin 7709f250f6 refactor: Move os.h/cpp and deps to lib_system
This is a prerequisite for the next objective: move logging into its own lib.
As logging is a core dependency for so much of our system, this will enable
us to move more complex services (like md providers) into smaller libs.
2024-09-18 14:02:40 +00:00
John Colvin dc31b4919d cleanup: Remove unused code 2024-09-18 14:02:40 +00:00
John Colvin 0f8f7f3adc cleanup: Remove more unnecessary includes 2024-09-18 14:02:40 +00:00
John Colvin eae4f571ee refactor: Rename InstallBasicConfig -> InstallBasic 2024-09-18 14:02:40 +00:00
John Colvin e85cdcb453 refactor: Move SecurityMarketDataStatistics to lib_domain 2024-09-18 14:02:40 +00:00
John Colvin 2c7f47ef17 cleanup: Remove unnecessary code 2024-09-18 14:02:40 +00:00
John Colvin b9f7d0b1a2 cleanup: Remove empty header FTXSecurity.h 2024-09-18 14:02:40 +00:00
John Colvin 7feed957a3 cleanup: Remove temporary redirect headers
These were used to make code migration smoother (to the new lib_system and lib_domain)

* spark/algorithm/TryCatch.h -> system/algorithm/try_catch.h
* spark/core/BoolName.h -> system/bool_name.h
* spark/core/EnumMacros.h -> system/enum_macros.h
* spark/core/FMT.h -> system/FMT.h
* spark/core/FMT_IMPL.h -> system/FMT_IMPL.h
* spark/core/TimeSpan.h -> system/time_span.h
* spark/core/TimeStamp.h -> system/time_stamp.h
* spark/domain/Price.h -> domain/price.h
* spark/io/JsonReader.h -> system/json/json_reader.h
* spark/io/JsonDumper.h -> system/json/json_dumper.h
2024-09-18 14:02:40 +00:00
John Colvin 14031da8eb refactor: Move RFQ to lib_domain 2024-09-18 14:02:39 +00:00
John Colvin 6fee0d99d9 refactor: Move AggressorSide proto to lib_domain
Get rid of non-proto AggressorSide enum while we're at it (consolidate to 1 type)
2024-09-18 14:02:39 +00:00
John Colvin 5db7a20a41 cleanup: Remove more unused code 2024-09-18 14:02:39 +00:00
John Colvin 250f7d4b6d refactor: Move a few Order types to lib_domain 2024-09-18 14:02:39 +00:00
John Colvin dd02b83bc1 refactor: Move TimeInForce to lib_domain 2024-09-18 14:02:39 +00:00
John Colvin 26c3af44e0 cleanup: Remove unused code 2024-09-18 14:02:39 +00:00
John Colvin 90a59b764e refactor: Move TradeRecord into lib_domain 2024-09-18 14:02:39 +00:00
John Colvin df750c4dd0 refactor: Move TradingStatus enum to lib_domain 2024-09-18 14:02:39 +00:00
John Colvin c35d7c90f6 build: Update webpack-cli in admin webapp 2024-09-18 14:02:39 +00:00
John Colvin 1c2dc2c39c build: Configure admin webapp build entirely in CMakeLists
Get rid of build_admin_web_app.py
2024-09-18 14:02:39 +00:00
John Colvin 3faffbe22a refactor: Move domain types into new lib_domain
Also rename files in snake_case (standard naming convention for protos)
https://protobuf.dev/programming-guides/style/
2024-09-18 14:02:39 +00:00
John Colvin 7864c3238c refactor: Move collections into lib_system 2024-09-18 14:02:39 +00:00
John Colvin cf4a8599e7 refactor: Move a handful of types/utils to lib_system 2024-09-18 14:02:39 +00:00
John Colvin 7459a4f224 build: Add preset to build with 6 jobs (-j 6)
Otherwise the devcontainer may hog too much cpu and cause the container to freeze
2024-09-18 14:02:39 +00:00
John Colvin f1451b102d cleanup: Remove unused includes
Mostly collection macro headers
2024-09-18 14:02:39 +00:00
John Colvin 23b2758a05 cleanup (build): Move protobuf service generation cmake into lib_spark
Because lib_spark owns all the services
2024-09-18 14:02:39 +00:00
John Colvin 5c3a57ceb5 refactor: Rename protobuf out dir "include" -> "protobufs"
This is a more accurate and precise name. The dir contains c++ code generated
from .proto files - both sources (.cc) and headers (.pb.h).
2024-09-18 14:02:39 +00:00
John Colvin 888a6bc536 refactor: Move ValueWrapper into lib_system
So other projects can use it
2024-09-18 14:02:39 +00:00
John Colvin 0514d8f434 Include bts_assert.h in bts_fatal.h
We shouldn't have to include bts_assert.h when we're already including bts_fatal.h
2024-09-18 14:02:39 +00:00
John Colvin cf0adfd910 refactor: Split MarketDataGateway up into providers/sources
MarketDataGateway still exists, but is greatly simplified. It creates
several `IMarketDataProvider`s, each of which generates a list of
`IMarketDataSource`s. The market data sources provide market data to an
`IMarketDataListener`.

* EmumbaMarketDataProvider
* CCApiMarketDataProvider
* PyMarketDataProvider
* OnixsMarketDataProvider
* ReplayMarketDataProvider

As a side effect, individual py market data sources are now explicitly
limited to a single venue (it probably should have been this way all
along, since the shared memory writers do not expect updates to involve
multiple venues).
2024-09-18 14:02:39 +00:00
John Colvin bfc87bfdf0 fix build: Remove leftover cmake setup
From before recent build refactoring.
2024-09-18 14:02:39 +00:00
John Colvin d9c8c61e47 fix build: Remove old install target
Leftover from before recent build refactoring.
2024-09-18 14:02:39 +00:00
John Colvin 3398f44b33 fix: Include GNUInstallDirs in InstallBasicConfig
We reference variables defined therein (otherwise the burden of
inclusion falls to the caller).
2024-09-18 14:02:39 +00:00
John Colvin f4cef5aadb cleanup: Remove superfluous code 2024-09-18 14:02:39 +00:00
John Colvin e12769df8e improvement: Set SHARED_MEMORY_PREFIX to an environment variable
So we can effectively define it at runtime. We don't want to have to
produce different builds just to change the shared mem prefix.
2024-09-18 14:02:39 +00:00
John Colvin 6e5ddb734f fix: Always prepend proc name to shared mem prefix
Our shared memory model isn't really designed to be shared across processes.
Maybe some day it will.
2024-09-18 14:02:39 +00:00
John Colvin b4345cfa24 cleanup: Remove redundant macro def 2024-09-18 14:02:39 +00:00
John Colvin 54566ade60 Skip md recording thread loop if no recorders configured to run 2024-09-18 14:02:39 +00:00
John Colvin 06a80040ff Handle signal interrupts explicitly in sparkd
Otherwise the python interpretter, which is instantiated when we load
python md plugins, prevents the application from exiting.
2024-09-18 14:02:39 +00:00
John Colvin 2e9b4c78f3 feat: Add support for python md plugins 2024-09-18 14:02:39 +00:00
John Colvin 2b62668df8 fix build: Ensure py scripts use unix line endings
I'm not sure why, but my sparkd build (linux) suddenly began failing to
generate cpp headers for protos. The error message indicated it was due
to an unexpected carriage return (\r) in plugin.py:

/usr/bin/env: ‘python\r’: No such file or directory
2024-09-18 14:02:39 +00:00
John Colvin 17ca432f87 refactor: Add lib_system proj for common utils
Migrates the following:
* Delegate.h
* bts_array_size.h
* string_utils.h

Note: File names in the new lib are snake_case (this seems to be the
standard naming convention for c++).
2024-09-18 14:02:39 +00:00
John Colvin 2c8833bdfe vscode: Fix sparkd path in launch.json 2024-09-18 14:02:39 +00:00
John Colvin e0ab5713b6 build: Clean up libspark CMakeLists 2024-09-18 14:02:39 +00:00
John Colvin c31b52e1df cleanup: Trivial whitespace cleanup in sparkd CMakeLists 2024-09-18 14:02:39 +00:00
John Colvin 2e7478629f build: Add cmake fns to symlink and install local libs
The code needed to do this (inline) is ugly and verbose
2024-09-18 14:02:39 +00:00
John Colvin 3afc447538 cleanup: Use conventional UPPER_CASE var names in sparkd CMakeLists 2024-09-18 14:02:39 +00:00
John Colvin 56887327e1 cleanup: Trivial whitespace cleanup in Protobuf.cmake 2024-09-18 14:02:39 +00:00
John Colvin 6c89cd6717 build: Remove explicit targets from sparkd build preset
So it builds the default targets (all). They were only listed explicitly
in the first place so we could build specific targets in vs code using
the dropdown
2024-09-18 14:02:39 +00:00
John Colvin 5243490d5f build: Set up necessary paths, symlinks for sparkd in sparkd cmake proj
Rather than performing this step in the root CMakeLists, as we previously did
in the "server_layout" target. This effectively replaces the "server_layout" target.
2024-09-18 14:02:39 +00:00
John Colvin 0d04eea483 build: Clean up data_files cmake proj (support --target clean) 2024-09-18 14:02:39 +00:00
John Colvin 3abfac1a20 fix build: Generate proto hdrs in "include" dir (child of proj's bin dir)
Reorganizing the sparkd proj with stuttering dirs broke the build because
the generated protos were being dumped into a dir with the same name
as the "sparkd" executable.
2024-09-18 14:02:38 +00:00
John Colvin 574694e06e fix build: Update sparkd & risk webapp after moving libspark 2024-09-18 14:02:38 +00:00
John Colvin c30bba7d5c refactor: Parse input args in Protobuf_Generate_cpp
So we can be more precise when generating protobufs. For example, now we
can specify different include dirs when generating code for different
group of proto messages - an obvious improvement over the previous
system, which relied on a hardcoded superset of include dirs that would
support every proto message in our code base.
2024-09-18 14:02:38 +00:00
John Colvin 48695d3f7f refactor: Orgnaize libspark proj w/ stuttering include 2024-09-18 14:02:38 +00:00
John Colvin 3ef40387f3 build: Add install targets for libspark 2024-09-18 14:02:38 +00:00
John Colvin 776625891e vscode: Add debug cfg to skip sparkd build 2024-09-18 14:02:38 +00:00
John Colvin fcdd869831 fix build: Add ShmSegmentName.cpp to windows client sources 2024-09-18 14:02:38 +00:00
John Colvin 6a7efacac9 Make a few timestamp fns const
They should have been const in the first place
2024-09-18 14:02:38 +00:00
John Colvin 489b289a6d Add config profiles to libspark (check pt)
This is part of a larger effort to unify the libspark build, and eliminate
compile time differences between the builds used by our various services
2024-09-18 14:02:38 +00:00
John Colvin e75cc05137 Add api server to docker compose 2024-09-18 14:02:38 +00:00
John Colvin f75fe665c1 Rename pingtest "sparkd-ping-test", make it depend on sparkd
Add it to a "ping-test" profile, as we may add similar tests for other services
2024-09-18 14:02:38 +00:00
John Colvin 58bd35be01 cleanup: Remove unnecessary env vars from webapp docker compose 2024-09-18 14:02:38 +00:00
John Colvin 89143b25c7 Change api server ports so they don't conflict w/ ds webapp 2024-09-18 14:02:38 +00:00
John Colvin aec005505f vscode: Fix sparkd debug config 2024-09-18 14:02:38 +00:00
John Colvin 103725887f build: Resolve app data dir and shared mem prefix at runtime
We really can't define them at compile time anymore as we build multiple
exes that depend on libspark in the same dir (e.g., spark api server,
pubsub, ds-publisher, webapp, etc.). That means one libspark.so (or dll)
must serve all exes. We should compile a consistent libspark to mitigate
the risk of running an exe with the wrong version (of libspark). For
example, running pubsub with a libspark configured for spark api server,
leftover from an earlier build.
2024-09-18 14:02:38 +00:00
John Colvin 1ce7ddd252 Don't log "initializing spark config" if it has already been initialized 2024-09-18 14:02:37 +00:00
John Colvin 3e1000e8e7 vscode: Add debug cfg for spark api server 2024-09-18 14:02:37 +00:00
John Colvin 5561c54345 Use dependency injection in spark api server 2024-09-18 14:02:37 +00:00
John Colvin c7269176ca Add SparkLoggerAdapter for Msft ILogger 2024-09-18 14:02:37 +00:00
John Colvin ec34e8b2f2 Display exception stack trace w/ 1 call per line in ds webapp 2024-09-18 14:02:37 +00:00
John Colvin a54f9530b9 Remove unused ErrorView from ds webapp 2024-09-18 14:02:37 +00:00
John Colvin 53850044c0 Propagate ViewStream errors to webapp frontend 2024-09-18 14:02:37 +00:00
John Colvin 8e91091e3c Add ErrorChanged event to propagate DataStream exceptions 2024-09-18 14:02:37 +00:00
John Colvin 92ffade5da Set columns to empty array rather than null in ds webapp
If no data has been received. Since the 'columns' property was marked
'required', this eliminates a warning that was being logged to console.
2024-09-18 14:02:37 +00:00
John Colvin b636cf76f2 Improve alignment/spacing in ds webapp 2024-09-18 14:02:37 +00:00
John Colvin ac001b65c6 Configure datastream deployment with docker-compose
To deploy to aws, connect to our aws vpn then run:
tools/deploy-docker-compose.sh

The deploy script does the following:
* builds sparkd
* builds datastream apps (pubsub, md-publisher, and webapp)
* uploads the build to aws
* bounces all services (sparkd, pubsub, md-publisher, and webapp)

Note: the name 'md-publisher' refers to DataStreamPublisher w/ MarketData
configuration
2024-09-18 14:02:37 +00:00
John Colvin bc3869f087 Remove "BTS Cloud Streams" title from webapp
Space is precious when viewing on phone (tbh, it didn't look that good
anyways).
2024-09-18 14:02:37 +00:00
John Colvin 66583356ac fix: Ensure md-publisher waits for pubsub and sparkd
Bugs (fixed):
  * md-publisher silently fails if it tries to connect before those services are "ready".

The Docker compose "depends_on" property ensures sparkd and pubsub
containers are up, but doesn't prevent startup race bugs (e.g., if
md-publisher and pubsub start around the same time, and md-publisher
tries to connect before pubsub has started accepting connections). To
address these race conditions, we use a nifty tool called "dockerize"

References:
  * https://github.com/jwilder/dockerize
2024-09-18 14:02:37 +00:00
John Colvin ef11c6e418 Replace ErrorView component w/ robust QueryStatusView 2024-09-18 14:02:37 +00:00
John Colvin 361e4bdd48 Fix typos in query button theme 2024-09-18 14:02:37 +00:00
John Colvin 44ad5861fc fix: Don't assert on exception in BooleanExpression.evaluate
Bugs (fixed):
  * DataStream WebApp Server sometimes crashes when the user enters a
    bad query.
2024-09-18 14:02:37 +00:00
John Colvin 0ea6563ddb Change VITE_DATASTREAM_WEBSOCKET_URL to localhost 2024-09-18 14:02:37 +00:00
John Colvin ff7f8fce49 Assert that local ip address is successfully obtained 2024-09-18 14:02:37 +00:00
John Colvin cc28ba41d1 improvement: Resolve hostnames when comparing ip addrs
So we can to configure pubsub routes by hostname or ipv4 (ipv6 hasn't been
tested yet...so we should assume it doesn't work).
2024-09-18 14:02:37 +00:00
John Colvin 6fcec2c370 Assert TcpConnection endpoint matches expected 2024-09-18 14:02:37 +00:00
John Colvin 76393216c1 Fix typo in log msg (was missing interpolation '$') 2024-09-18 14:02:37 +00:00
John Colvin 3e792a41e0 Catch exceptions on AsyncTCPConnector_AsyncWrapper background thread
Since it spawns a new thread on each attempt, uncaught exceptions were previously undetected,
and caused silent failures down the pipe. This should make it easier to track such bugs down.
2024-09-18 14:02:37 +00:00
John Colvin 298273de91 fix: Format PubSubRoute as json in log msgs
It was previously using the default ToString, which simply returns the type name
2024-09-18 14:02:37 +00:00
John Colvin d256ec508b build: Upgrade packages (deps) in DataStreamTests 2024-09-18 14:02:37 +00:00
John Colvin 8293f1764e cleanup: Remove unused ViewStreamDatabase config 2024-09-18 14:02:37 +00:00
John Colvin 0a119b3f03 vscode: Add PubSubServer debug cfg to skip build 2024-09-18 14:02:37 +00:00
John Colvin e4940bba40 Configure apps as a distributed system via Docker compose 2024-09-18 14:02:37 +00:00
John Colvin 6b2a86f4f1 Remove reference to spark protobuf Config in ViewStream
We can't read spark protobuf Config until it has been initialized,
and since ViewStream doesn't have the configServer param handy it
isn't able to initialize config if necessary. In that case, it would simply
block until config was initialized by someone else. Such a risk doesn't
seem justified for this particular configuration dependency
(DataStreamConfig.UseUncopiedNewRowsInFire). For now, its hardcoded to false.
2024-09-18 14:02:37 +00:00
John Colvin 01b9d49df2 refactor: Run RemoteQueryManager as an IHostedService
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2024-09-18 14:02:37 +00:00
John Colvin f3d7a9210d Remove inappropriate "Unable to dispose subscription" log msg 2024-09-18 14:02:37 +00:00
John Colvin 32e766f56c fix: Ensure spark config init and login are only performed once
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.

It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2024-09-18 14:02:37 +00:00
John Colvin 58557311d4 Move misplaced log abt initializing query text to appropriate place 2024-09-18 14:02:37 +00:00
John Colvin 412a2576e0 Assert ViewStreamDatabase.instance is only created once
The instance must be initialized via ViewStreamDatabase.SetGlobalInstance
2024-09-18 14:02:37 +00:00
John Colvin 06de560169 cleanup: Remove unused code from ViewStreamDatabase 2024-09-18 14:02:37 +00:00
John Colvin 80b62a5a33 Include subscription id in ds websocket updates 2024-09-18 14:02:37 +00:00
John Colvin f05cff4ada fix: Define BoooleanConstant before StringConstant
Otherwise the former masks the latter and prevents it from matching
2024-09-18 14:02:37 +00:00
John Colvin b2d04363b6 Log event data in websocket open, close, error callbacks 2024-09-18 14:02:37 +00:00
John Colvin d9d2f70917 refactor: Bring back Material DataGrid
Fix the column resize bugs that made it previously unusable
2024-09-18 14:02:37 +00:00
John Colvin 3d07e8cf3d refactor: Improve react webapp state management
* Add queryTextResetTime as part of the CannedQuery state
  (the QueryControl compares the canned query click time with the time of the
  last user input, and applies the text from the most recent change)

* Replace 'isRunning' (bool) with a 'queryState' (string) to reflect the subscription status
  value received from the server. Less indirection this way.

* Replace inappropriate useMemo call w/ useEffect
2024-09-18 14:02:37 +00:00
John Colvin 68f75a6f44 cleanup: Get rid of magic const (subscriptionId) in webapp App.jsx 2024-09-18 14:02:37 +00:00
John Colvin ad0b6968ad build: Copy configuration profiles to publish dir 2024-09-18 14:02:37 +00:00
John Colvin fdaff5dcf5 cleanup: Remove unnecessary deps from csprojs 2024-09-18 14:02:37 +00:00
John Colvin ffa92ea4ae build: Change publish_data_stream.py to publish the sln
We were publishing individual projects as a workaround for the
dependency issue (bug) that caused some projects to incorrectly target
net8.0. The fact that publishing individual projects "just worked" was
probably dumb luck. Now that the underlying issue has been addressed, we
can resume publishing the sln as a whole.
2024-09-18 14:02:37 +00:00
John Colvin dd305b9c3f dev: Install docker in devcontainer
So we can start various services like sparkd, pubsub, ds publishers, ds
webapp, etc. from inside the devcontainer.
2024-09-18 14:02:37 +00:00
John Colvin 07b581e980 dev: Remove dotnet8 from devcontainer
We'll upgrade to dotnet8 eventually, but now isn't the time (too much
more important work to be done).
2024-09-18 14:02:37 +00:00
John Colvin 79fadf3dba dev: Run devcontainer on host network
So we don't need to worry about publishing the right ports (and don't
need to rebuild the container everytime the list of ports changes).
2024-09-18 14:02:37 +00:00
John Colvin c05d90fd26 fix: Downgrade Microsoft.Extensions.Hosting to 7.0.1
A handful of csprojs were referencing version 8.0.0. Since that version
targets net8.0, I have no idea why it didn't cause the build to fail.
Instead, this framework mismatch led to a mysterious runtime crash with
the following error, despite the dll in question being present in the
build dir:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.
The system cannot find the file specified.

It is worth noting that this crash would only happen for exes published
as part of the DataStreamSuite.sln with the "--self-contained" flag. For
example, the Bts.DataStreamSuite.Server exe would always fail if
published via:

dotnet publish --configuration Release --sc --framework net7.0 --runtime linux-x64 <DataStreamSuite.sln>
2024-09-18 14:02:37 +00:00
John Colvin 51f60188f5 Add HttpsRedirection middleware first in the webapp pipeline
Middleware handlers are invoked in the order in which they are added. Any middleware
that is added before HttpsRedirection may can prevent the request from being redirected.
As of now, only development-specific middleware is added ahead of HttpsRedircetion.
2024-09-18 14:02:37 +00:00
John Colvin cf93176991 build: Copy configuration profiles to build dir
So they are readily available to publish/deploy scripts.
2024-09-18 14:02:37 +00:00
John Colvin 6aa9307a3c Fix benign typo in Scanner/Parser msbuild targets
The output path referenced a non-existant msbuild metadata prop "RelativeOutputDir"
(the actual prop is called "RelativeDir"). Also, the OutputFile path is relative to the
project dir (the 'MSBuildProjectDir' prop), not the file being included.
2024-09-18 14:02:37 +00:00
John Colvin 3eb3cfbf9f fix: Catch exception when creating TcpClient 2024-09-18 14:02:37 +00:00
John Colvin 501671c9c5 fix: Catch WebSocketException in DataStreamSubscription 2024-09-18 14:02:37 +00:00
John Colvin 9d62fcde3d fix: Add null check to NonIncGroupByStream 2024-09-18 14:02:37 +00:00
John Colvin 50b4c8807e fix: Add null check to TableAccumulator2 2024-09-18 14:02:36 +00:00
John Colvin c6f6dc8bf4 fix: Add self-signed certificate to Dockerfile for ds webapp 2024-09-18 14:02:36 +00:00
John Colvin fb528d4683 fix: Make SystemSubscriptionMapPublisher threadsafe
The PubSubState.onTagMapChanged event may not be invoked on
the same thread as doWork.
2024-09-18 14:02:36 +00:00
John Colvin f1b2206577 fix: Add null check in TimeBasedFilter.cs 2024-09-18 14:02:36 +00:00
John Colvin 68d1865f36 Reduce task wait/timeout code in AsyncTCPConnector 2024-09-18 14:02:36 +00:00
John Colvin 3c8431837b Add a few canned queries to AwsProd.json for ds webapp 2024-09-18 14:02:36 +00:00
John Colvin e0d413b884 fix: Copy canned queries to clipboard on click
Instead of trying to update the text directly, which causes the front end
to overwrite the user's changes everytime the data updates from then on.
2024-09-18 14:02:36 +00:00
John Colvin ff8ebaa326 fix: Convert invalid doubles to nan in ds webapp serializer 2024-09-18 14:02:36 +00:00
John Colvin 370b84b6dd Increase datagrid maxheight in ds webapp to 1000px 2024-09-18 14:02:36 +00:00
John Colvin 715246c9c7 Improve error display in ds webapp 2024-09-18 14:02:36 +00:00
John Colvin a1f195721a fix: Dispose existing sub on subscribe req in ds webapp
Even if new sub has parse error. This fixes a bug where the previous
subscription would continue firing and updating the frontend, when it
should just display the error info to the user and cease updates.
2024-09-18 14:02:36 +00:00
John Colvin d7de423e35 fix: Get rid of "UseSpa" calls in webapp Production environment
This was leftover from a misguided attempt to fix a bug that turned out
to be due to an omitted environment var:
ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=Microsoft.AspNetCore.SpaProxy
2024-09-18 14:02:36 +00:00
John Colvin 244b3de513 Define webapp specific env vars in generic datastream app deploy script
For now, this makes things easier, as I don't need to write a special script just for the webapp
2024-09-18 14:02:36 +00:00
John Colvin 0683abbda6 Add config profile and Dockerfile args to deploy script 2024-09-18 14:02:36 +00:00
John Colvin 9d2886fe67 Copy DataStream WebApp 'dist' to 'wwwroot' in deploy script 2024-09-18 14:02:36 +00:00
John Colvin 2ac815949c Tweak DataGrid style in DataStream WebApp 2024-09-18 14:02:36 +00:00
John Colvin 046f451e76 Add AwsProd config for DataStream WebApp 2024-09-18 14:02:36 +00:00
John Colvin 36f191d49a fix: Add VITE_ prefix to imported env vars
For the DataStream WebApp client
2024-09-18 14:02:36 +00:00
John Colvin f55237029f Fix urls for DataStreamPublisher AwsProd.json 2024-09-18 14:02:36 +00:00
John Colvin 6fcf096924 Add benzinga news streams to AwsProd.json
This is a config profile for DataStreamPublisher, designed to be deployed
to aws (on public internet).
2024-09-18 14:02:36 +00:00
John Colvin 547383c8d9 fix build warnings: Make props nullable strings in SharedStreamConfig 2024-09-18 14:02:36 +00:00
Taha dc74b62304 Added benzinga streamd to MDPub's test config 2024-09-18 14:02:36 +00:00
Taha Afzal 8907681bb2 PubSubSingleTagState is not needed any more 2024-09-18 14:02:36 +00:00
Taha f16b921630 Icomplete SharedStreamsConfig was breaking build, fixing 2024-09-18 14:02:36 +00:00
Taha 614e9c69e1 Added new benzinga streams 2024-09-18 14:02:36 +00:00
John Colvin 5a59303dd4 whitespace: Format PubSubServer.json (indentation issues) 2024-09-18 14:02:36 +00:00
John Colvin d427538cd9 fix import from environment variable 2024-09-18 14:02:36 +00:00
John Colvin 46888af8fb Add a few tasks to run DataStream WebApp 2024-09-18 14:02:36 +00:00
John Colvin efaef11e7c fix add scripts to deploy data stream apps 2024-09-18 14:02:36 +00:00
John Colvin c095614c67 fix: Import websocket url from environment var 2024-09-18 14:02:36 +00:00
John Colvin f41114ee13 Add scripts to deploy data stream apps 2024-09-18 14:02:36 +00:00
John Colvin 39c33f082f Add SpaServer to Production environment
For Bts.DataStreamWebApp.Server
2024-09-18 14:02:36 +00:00
John Colvin a3c0bc977e Change DirectoryBrowser requestPath to "/browse"
So we can view a psuedo file explorer for the webapp by requesting the "/browse" path
(only activated in Development environment).
2024-09-18 14:02:36 +00:00
John Colvin 3a39b3c516 Publish Bts.DataStreamWebApp.Server along with PubSub & DataStream 2024-09-18 14:02:36 +00:00
John Colvin 9265746198 cleanup: Remove unused code in Bts.DataStreamWebApp.Client 2024-09-18 14:02:36 +00:00
John Colvin 0236e3a47a Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2024-09-18 14:02:36 +00:00
John Colvin 71d52c2dc9 Hardcode webSocketUrl to aws server in DataStreamWebApp client 2024-09-18 14:02:36 +00:00
John Colvin ce0b4872ff Assert that System publishers 'doWork' is always called on same thread 2024-09-18 14:02:36 +00:00
John Colvin b686159cf3 fix: Make InternalPublisherQueue thread safe 2024-09-18 14:02:36 +00:00
John Colvin ee7c8c393e Improve error reporting in DataStreamWebSocketServer 2024-09-18 14:02:36 +00:00
John Colvin 7c4191289f data stream web app client check pt 2024-09-18 14:02:36 +00:00
John Colvin f18949a1e3 fix build: Fix Spark build after making config init async 2024-09-18 14:02:36 +00:00
John Colvin f3b9cb2d62 fix Change Configuration.initialize to be async 2024-09-18 14:02:36 +00:00
John Colvin faeb10cf9d refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2024-09-18 14:02:36 +00:00
John Colvin e5f1039671 refactor: Change Configuration.initialize to be async 2024-09-18 14:02:36 +00:00
John Colvin 39de6bdb2d vscode: Add 'isSparkApiEnabled' input to launch.json 2024-09-18 14:02:36 +00:00
John Colvin e93f970706 SparkApiService check pt 2024-09-18 14:02:36 +00:00
John Colvin 844677dae0 pubsub server checkpt 2024-09-18 14:02:36 +00:00
John Colvin 3541feef87 pubsub checkpt 2024-09-18 14:02:36 +00:00
John Colvin 776a589945 datastreamwebapp server checkpt 2024-09-18 14:02:36 +00:00
John Colvin 396f72d7c1 datastream webapp client check pt 2024-09-18 14:02:35 +00:00
John Colvin 62a9917a48 fix add pubsub services 2024-09-18 14:02:35 +00:00
John Colvin a3dd7c3815 Remove unused PubSubConfig members from MarketDataPublisher cfg json 2024-09-18 14:02:35 +00:00
John Colvin e669cf6b77 Remove unused PubSubConfig members from PubSubServer cfg json 2024-09-18 14:02:35 +00:00
John Colvin c5e3c4aa18 fix Remove unused PubSubConfig members 2024-09-18 14:02:35 +00:00
John Colvin be35ed120b Remove unused PubSubConfig members from DSQA config json 2024-09-18 14:02:35 +00:00
John Colvin 9846bf3936 Remove unused PubSubConfig members from WebApp config json 2024-09-18 14:02:35 +00:00
John Colvin cb5fcee2ce Remove unused PubSubConfig props 2024-09-18 14:02:35 +00:00
John Colvin 08319cfd38 Fix dsqa config json 2024-09-18 14:02:35 +00:00
John Colvin 777c276437 fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2024-09-18 14:02:35 +00:00
John Colvin 5ab0e1f886 improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2024-09-18 14:02:35 +00:00
John Colvin a90896f606 Fix mismatched types in PubSubServerTagStatsStream schema 2024-09-18 14:02:35 +00:00
John Colvin 3f6f5ab0e0 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-09-18 14:02:35 +00:00
John Colvin 24e5683ad2 refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-09-18 14:02:35 +00:00
John Colvin 34999d0a5a Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-09-18 14:02:35 +00:00
John Colvin ace7ae2695 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-09-18 14:02:35 +00:00
John Colvin f9016907e6 refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2024-09-18 14:02:35 +00:00
John Colvin 0913fe28d4 refactor: Only support 1 tag type in IPubSubServerTagHandler 2024-09-18 14:02:35 +00:00
John Colvin 400b4ff848 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2024-09-18 14:02:35 +00:00
John Colvin 5a36a5f4d4 improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-09-18 14:02:35 +00:00
John Colvin 0f6539ed00 fix: Dispose market data sub in various md streams 2024-09-18 14:02:35 +00:00
John Colvin 475bf1effc cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-09-18 14:02:35 +00:00
John Colvin ec1e657687 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-09-18 14:02:35 +00:00
John Colvin 284c84dfe6 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-09-18 14:02:35 +00:00
John Colvin b78c61bf44 fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-09-18 14:02:35 +00:00
John Colvin 09a41775bb build: Add scripts to publish data stream apps 2024-09-18 14:02:35 +00:00
John Colvin b9ef3ddf91 Fix typo in log msg 2024-09-18 14:02:35 +00:00
John Colvin e516496fe8 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-09-18 14:02:35 +00:00
John Colvin a55705db75 vscode: Add DataStreamPublisher debug cfg to skip build 2024-09-18 14:02:35 +00:00
John Colvin e39ad91e1c build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-09-18 14:02:35 +00:00
John Colvin 658472b2a8 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-09-18 14:02:35 +00:00
John Colvin 1ad94cfe49 Add spark service modules designed for Dependency Injection 2024-09-18 14:02:35 +00:00
John Colvin 22a5a9a4c1 build: Copy libspark to all dotnet dirs in single cmake target
This is cleaner than the previous impl, which used separate build
targets to copy libspark to "net7.0", and "net7.0-windows" (dirs).
2024-09-18 14:02:35 +00:00
John Colvin e5bbc26da5 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-09-18 14:02:35 +00:00
John Colvin 6a628a3070 fix build (api server): Change namespace to bts.websocket_core 2024-09-18 14:02:35 +00:00
John Colvin ac120658eb Remove docker submodules again 2024-09-18 14:02:35 +00:00
John Colvin 85a36384ce Change externals url from ssh to https style
So we don't need ssh keys to clone it (the builtin credential manager
makes https authentication safe/easy)
2024-09-18 14:02:35 +00:00
John Colvin 563833bc67 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-09-18 14:02:35 +00:00
John Colvin d77ac1600b fix build (warn): Only use Registry on windows 2024-09-18 14:02:35 +00:00
John Colvin 3b41a2065a cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-09-18 14:02:35 +00:00
John Colvin f993f01634 cleanup: Normalize indentation in TradeBarsStream 2024-09-18 14:02:35 +00:00
John Colvin e7067a708b cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-09-18 14:02:35 +00:00
John Colvin 04ac2b9d3d cleanup: Modernize PubSubRoute code, json parsing 2024-09-18 14:02:35 +00:00
John Colvin 66c6b0bf8c cleanup: Remove unused DateTime grammar from exp parser 2024-09-18 14:02:34 +00:00
John Colvin 03dd4000fe Add JsonRpcRequestObservable, AsyncEnumerable 2024-09-18 14:02:34 +00:00
John Colvin e928295e71 Add json facilities for DataStream websocket api 2024-09-18 14:02:34 +00:00
John Colvin b94c5823e9 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-09-18 14:02:34 +00:00
John Colvin 467e81b543 feat: Add JsonRpc types for websocket paradigm 2024-09-18 14:02:34 +00:00
John Colvin 8f7409b3e7 cleanup: Remove empty file (clearly added by accident) 2024-09-18 14:02:34 +00:00
John Colvin 0d19702e12 fix: Fix improper 'OfType' linq cast
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2024-09-18 14:02:34 +00:00
John Colvin 466fa265fe editorconfig: Use standard c# capitalization rules
* Use 'PascalCase' for public members (instead of 'snake_case')
* Use 'camelCase' for private members (instead of '_snake_case')
2024-09-18 14:02:34 +00:00
John Colvin ebd1852221 refactor: Add WebsocketClient class
Basically an ISubject (I/O) wrapper for IWebsocketConnectionGateway
2024-09-18 14:02:34 +00:00
John Colvin f1130160b1 refactor: Change WebsocketServerConfig.port to uint32
It should always be a number between 1-65535
2024-09-18 14:02:34 +00:00
John Colvin 99fba0b3ce refactor: Rename WebsocketServerConfig proto pkg 'bts.websocket_core'
This matches the namespace used in spark_websocket csproj. Eventually,
I'd like to use conventional capitalization (TitleCase namespaces/types,
etc.), but let's shelf that until we have enough time/energy to do a
more comprehensive refactoring.
2024-09-18 14:02:34 +00:00
John Colvin 89afbbd00a refactor: Remove nullability constraint in GetValueOrDefault 2024-09-18 14:02:34 +00:00
John Colvin 9e4af550f8 refactor: Use events to avoid circular deps for ws sync/gateway 2024-09-18 14:02:34 +00:00
John Colvin ea67280252 refactor: Let caller specify nullability of T in json parse fn 2024-09-18 14:02:34 +00:00
Taha Afzal 8c06d84a8b Fixing problems where PSS's client socket's IP was different than what the client was sending in PubSubClientSubscription 2024-09-18 14:02:34 +00:00
Taha Afzal 941186a12a Fixing PubsubEchoTestStream 2024-09-18 14:02:34 +00:00
Taha Afzal 0984007c6e Added pss_echo() 2024-09-18 14:02:34 +00:00
Taha Afzal 2bb7a89777 Merging 2024-09-18 14:02:34 +00:00
Taha Afzal 15fc3cb785 src/DataStreamSuite/PubSub/ManagedTCP/AsyncTCPConnection_AsyncWrapper.cs 2024-09-18 14:02:34 +00:00
Taha 725396589e Adding TableMetaDataStream 2024-09-18 14:02:34 +00:00
Taha Afzal 6fc1ec4d67 Cleaning Config 1.0 (not completely done) 2024-09-18 14:02:34 +00:00
Taha 0496b299dd Added pubsub routes to the Title of DSQA 2024-09-18 14:02:34 +00:00
Taha b55c4103df Fixing the bug where messages were being sent by the pubsubserver out of order 2024-09-18 14:02:34 +00:00
Taha 6604eaaea7 some buf gixes + cleanup 2024-09-18 14:02:34 +00:00
Taha 29e5c72969 PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop 2024-09-18 14:02:34 +00:00
Taha 21e07e1b2e PubSubServer's linux issues fixed 2024-09-18 14:02:34 +00:00
John Colvin 74afdde8de fix build: Add ".list" to SymbolLists config
For new exchanges, TAIFEX and CBOE
2024-09-17 13:01:05 +00:00
John Colvin 0a92ad2eb1 fix (build): Add break statement to new venue switch cases
Namely, TAIFEX and CBOE
2024-09-17 13:00:14 +00:00
John Colvin 56d207c3f1 fix: Preserve vs exp sec type (Option) when computing implied vol
Because the exp keys in the data map have sec type Option.
2024-09-16 13:46:53 -04:00
John Colvin 9172e752e3 ui: Highlight launcher buttons when selected
Namely, the expandable menu buttons in Analytics shell module
2024-09-16 13:46:53 -04:00
John Colvin adc9f2acfc Replace trivial refs to ThreadedBlackboard w/ IBus 2024-09-15 23:45:15 -04:00
John Colvin 41d17e67d2 fix: Move accounts subscription to dispatcher thread
We update a collection that is bound to the view in the callback (mutation
ops throw if performed on a different thread).
2024-09-15 23:45:15 -04:00
John Colvin 5cacff073b fix: Check 'HasTimestamp' before accessing prop in man fill row
Pending adds (i.e., unpersisted new rows) don't have timestamps.
In such cases, the 'Timestamp' getter throws an exception.
2024-09-15 23:45:15 -04:00
John Colvin c8da92c250 Cache/index options for faster lookups in SystemContractData 2024-09-15 23:45:15 -04:00
John Colvin c4e76076e3 fix: Include venue w/ symbol when publishing interp greeks 2024-09-15 23:45:14 -04:00
John Colvin 6a715b9a68 Fix typo in comment 2024-09-15 23:45:11 -04:00
John Colvin 1f28940c95 fix: SelectorList.RemoveAt doesn't need 'convert_back' delegate
So don't throw an exception if it is null.
2024-09-15 23:45:11 -04:00
John Colvin ebff40b88f fix: Don't assume venue is in spread def string
Pass it as an argument to the parse fn. This should be fine for now, but
obviously a singular venue param won't work for cross exchange spreads.
2024-09-15 23:45:11 -04:00
John Colvin a6d14a3ac3 improvement: Move system contracts sub off of dispatcher thread
Also add 200ms throttle. Contracts shouldn't change that often, so we don't need
lightening speed here. Furthermore, such changes may trigger expensive cpu ops.

Note: this change also replaces refs to ThreadedBlackboard w/ IBus (the
former is somewhat tethered to the dispatcher thread, particularly by
its timer_callback).
2024-09-15 23:45:11 -04:00
John Colvin af96bcb9e3 fix: Pass symbol/venue from vs usage cfg to AnalyticsService proto 2024-09-15 23:28:59 -04:00
John Colvin 619702cb04 cleanup: Remove unused code 2024-09-15 23:28:59 -04:00
John Colvin 5baec19aff refactor: Index vol and theos data by symbol/venue tuple
Rather than just symbol, to avoid conflicts when the same symbol exists
at multiple exchanges
2024-09-13 10:50:23 -04:00
John Colvin 39ef30c1e3 cleanup: Remove redundant type VolSurfaceSymbolStructure
There is an identical type (in terms of prop types and corresponding proto indices)
called VolSurfaceSymbolRelationship that is used in more places.
2024-09-13 10:50:23 -04:00
John Colvin c2f2d28914 Move nested vm classes into own files
VolSurfaceSettingsEditor has too much code (still). It is hard to
navigate.
2024-09-13 10:50:23 -04:00
John Colvin e06bc3bdf5 Simplify driver symbol structure in vs settings vm
The code was confusing and difficult to follow.
2024-09-13 10:50:23 -04:00
John Colvin e19c19c013 fix: Raise prop changed event when is_selected changes 2024-09-13 10:50:23 -04:00
John Colvin 8f324bc507 Add venue to vs calc fns 2024-09-13 10:50:23 -04:00
John Colvin 12107c42ce fix: Provide venue param w/ symbol when computing fut slide 2024-09-13 10:50:23 -04:00
John Colvin 91c6fb4e52 diag: Log added/removed exps when syncing vs input w/ system exps
This shouldn't happen often, but if it does, we should know about it.

PS: We recently fixed a bug caused by inequalities between
VolSurfaceExpirations in vs input, and in SystemContractData. User
inputs to the vs grid would be immediately reverted, as the exps
associated with the inputs were not found in the SystemContractData.
This bug would have been easier to track down with this logging in
place.
2024-09-13 10:50:23 -04:00
John Colvin 7044355547 Add venue to VolSurface(DisplayGrid)Expiration
So we can handle symbol/exp date conflicts for securities that differ
only by venue (exchange)
2024-09-13 10:50:23 -04:00
John Colvin 583ae65fb2 Clean up VolSurfaceExpiration
* Enable nullable lang feat
* Use file-scoped namespace
* Sort members
* Include sec type in Compare impl
2024-09-13 10:50:22 -04:00
John Colvin 552fec0edc Add null check in SelectorList ctr 2024-09-13 10:50:22 -04:00
John Colvin fcf6927fbf cleanup: Remove unused code 2024-09-13 10:50:22 -04:00
John Colvin 23f6d827d5 refactor: Replace ThreadedBlackboard w/ IBus
In SingleVolSurfaceCommittedOutputRetriever
2024-09-13 10:50:22 -04:00
John Colvin f083d0abab Clean up VolSurfaceTimeParameters
* Enable nullable lang feat
* Use file-scoped namespace
* Implement ICloneable
2024-09-13 10:50:22 -04:00
John Colvin a895e36922 cleanup: Remove unnecessary ternary op 2024-09-13 10:50:22 -04:00
John Colvin 59d7a72288 fix: Actually use (sec) type arg in to_security_expiration 2024-09-13 10:50:22 -04:00
John Colvin 7731f2f4d1 fix: Check security_type in vs exp Equals impl 2024-09-13 10:50:22 -04:00
John Colvin 41ae5897c0 fix: Maintain sec type when converting SecurityExpiration to vs exp
Avoids data loss
2024-09-13 10:50:22 -04:00
John Colvin c7949ccfba refactor: Get rid of sec type 'Option' default (magic const)
Better to be explict
2024-09-13 10:50:22 -04:00
John Colvin 567af79c7f Fix selection binding issues in vs settings editor
This specifically addresses issues in the driver symbol grids.
Unfortunately, we can't bind directly to ComboCheckBox.SelectedItems
because the view is contained in TabControl. When the selected tab
changes, the previously selected tab clears its ItemsSource (and
consequently the SelectedItems) as a side effect of being unloaded.
2024-09-13 10:50:22 -04:00
John Colvin 6bb4a7b25c refactor: Enable nullable (ref types) in ImmutableListExtensions 2024-09-13 10:50:22 -04:00
John Colvin bc7cef4c34 Fix typo in namespace bts.collection -> bts.collections 2024-09-13 10:50:22 -04:00
John Colvin ab9dc361d6 fix: Use ComboCheckBoxItem.IsSelectedProperty
Instead of Selector.IsSelectedProperty. Unlike ComboBoxItem, ComboCheckItem's
IsSelectedProperty isn't linked to Selector.IsSelectedProperty.
2024-09-13 10:50:21 -04:00
John Colvin 39da0a4dde Add currency column to mview 2024-09-13 10:50:21 -04:00
John Colvin 44e864d697 fix: Use SetCurrentValue instead of SetValue
So we propagate values to/from the binding data source,
rather than overwrite the data source itself.
2024-09-13 10:50:21 -04:00
John Colvin 0a02da3aba Add venue, sec type dropdown filters to exp params ui 2024-09-13 10:50:21 -04:00
John Colvin 836771a7f0 fix: Add null check in ComboCheckBox.SelectedItemsCollectionChanged 2024-09-13 10:50:21 -04:00
John Colvin f386badcfb fix: Disconnect old INotifyCollectionChanged source when dp changes 2024-09-13 10:50:21 -04:00
John Colvin 1d43a484db fix: Index opt/fut exps by venue in vs analytics 2024-09-13 10:50:21 -04:00
John Colvin 91042cb41e fix: Support multiple exchanges in exp params view/vm 2024-09-13 10:50:21 -04:00
John Colvin 10c0e4247d cleanup: Remove unused 'option_symbol' param from compute_atm_data 2024-09-13 10:50:21 -04:00
John Colvin 8e34264e6d fix: Filter by exch when getting atm data for sheets
This is necessary now that we support exchanges with overlapping securities
2024-09-13 10:50:21 -04:00
John Colvin 0a840d256b fix: Remove sec id prop from SecurityExpiration
This class isn't designed to represent a particular contract; it's
basically a key to get option theos.
2024-09-13 10:50:21 -04:00
John Colvin 8a69c51cac refactor: Add 'onDispatcher' param to 'ThreadedBlackboard.subscribe'
This will allow us to gradually move subscriptions off of the dispatcher
(render) thread. When the render thread is pegged, the ui is laggy and
unresponsive. The new param defaults to true, so it won't change
anything until we explicitly set it to false in 'subscribe' calls.
2024-09-13 10:50:21 -04:00
John Colvin 0262fd40bc fix: Remove redundant SparkTimer threads
This addresses some slowness on the front end. Prior to this commit,
each ThreadedBlackboard would start its own timer thread. Since multiple
ThreadedBlackboard run concurrently, this multiplied the frequency of
the SparkTimer callbacks by the number of active ThreadedBlackboards.

The fix is to start this thread in an IShellDaemon rather than the
ThreadedBlackboard. This avoids duplicate threads because the former is
guarenteed to be instantiated only once, via dependency injection.
2024-09-13 10:50:21 -04:00
John Colvin 859dfe3544 refactor: Simplify FakeMDDaemon code that delays post
Use Task.Delay/ContinueWith instead of subscribing to the SparkTimer tag.
2024-09-13 10:50:21 -04:00
John Colvin 9c624a2f79 refactor: Use IBus instead of ThreadedBlackboard
This should address some slowness/mouse lag in the front end (recently
reported by Brandon at Immutable Cap)
2024-09-13 10:50:20 -04:00
John Colvin dec1061b40 fix: Detect and log ccapi depth msg parse errors
This fixes a crash that would happen if a depth item's price or quantity
were an empty string (now we log an error and keep going).
2024-09-13 10:50:20 -04:00
John Colvin be4895faa6 fix: Actually store currency in shared mem
This field has always been in the Security proto, though it until now it
had been omitted from the shared memory representation (FixedSecurity).
2024-09-13 10:50:20 -04:00
Mohammad Arham 3c9755f0dd ADD: Added delay to databento file reader 2024-09-12 15:10:34 +00:00
Mohammad Arham 61ed847d7d Add: Replay speed added to mdreplayer 2024-09-11 06:50:37 +00:00
Mohammad Arham fc9094fea8 Added: CBOE working on databento 2024-09-11 06:17:26 +00:00
Mohammad Arham e36c0efc4b Added: GLBX working on databento 2024-09-11 06:16:42 +00:00
Mohammad Arham 4158a14cff Added: option to send timestamp in onbook IMDListener 2024-09-11 06:15:40 +00:00
Mohammad Arham 201a45cba0 taifex deo 2024-09-05 00:59:00 +00:00
Mohammad Arham ab31b9b253 Add: Multiple files working taifex 2024-09-03 15:15:55 +00:00
Adeel Khan 780ddc79a7 Add: Taifex I083 Message 2024-09-03 20:06:48 +05:00
Mohammad Arham d5a2adaed6 fix: removed errernous code 2024-09-03 13:09:54 +00:00
Mohammad Arham 008ab3f068 Merge branch 'development.new_exchanges' of bitbucket.org:bluetradesys/spark into development.new_exchanges 2024-09-02 16:31:14 -04:00
Mohammad Arham 7d926f588d FIX: typo, changed date to day 2024-09-02 16:31:10 -04:00
Mohammad Arham 707cd07876 Add: CBOE L1 books and contracts working 2024-08-28 16:22:51 +00:00
Mohammad Arham f7ecb9090c add: code to start working on cboe 2024-08-27 12:33:23 +00:00
Mohammad Arham a19e8a7315 change: changes to make taifex work 2024-08-27 07:45:59 -04:00
John Colvin cf189ecf81 refactor: Start cboe and taifex md 'start_databento'
Similar to how we start all ccapi implemented exchanges in "_start_cc"
(in MarketDataGateway)
2024-08-26 19:13:59 +00:00
John Colvin 4405b7adcb refactor: Rename cc_md_listeners -> venue_listeners
They are general purpose, not specific to ccapi.
2024-08-26 19:09:57 +00:00
John Colvin e51ee7b7b9 refactor: Wrap IDGenerator and IMarketDataListeners in shared_ptrs
Our various market data providers (ccapi, emumba, replay, onixs)
dereference pointers to common instances of these types concurrently on
different threads. Although we've correctly managed memory ownership
thus far, we really should be using smart pointers (this is why they
were made).
2024-08-26 19:03:29 +00:00
John Colvin dfaa74f426 build: Add databento dep via FetchContent
Now we actually build libdatabento in the devcontainer along with our
code (as we do with the vcpkg deps, though this is not part of vcpkg)
2024-08-26 17:13:51 +00:00
Mohammad Arham 1d2ce59009 add: added taifex entries 2024-08-22 23:13:05 +00:00
Adeel Khan 3602cc05bb added: TaifexDataHandler, temp 2024-08-23 02:06:47 +05:00
Adeel Khan ac88a81ac6 Security Extensions 2024-08-23 01:42:24 +05:00
Adeel Khan 01fa8e0c63 added: files for Taifex, Cboe 2024-08-23 00:57:40 +05:00
John Colvin 59547e0f37 fix build: Lock fmt version to 10.2.1 in vcpkg
This fixes the front-end (Windows) libspark build
2024-08-21 13:45:43 -04:00
John Colvin 3ce100a9cf build: Add databento to sparkd
* Install databento deps in docker
  - libzstd-dev
* Add databento cmake module to externals
2024-08-21 13:49:53 +00:00
John Colvin 6513fb0cd5 cleanup: Remove unused include 'date.h' from MDSource
This was an external header, and may interfere with a dependency in the
databento lib (which we are about to add).
2024-08-21 13:49:24 +00:00
Taha Afzal fd0fb1f760 Adding incremental book updates to IMarketDataListener 2024-08-12 22:49:27 -04:00
Taha 5cf7eacd82 Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe 2024-06-24 00:28:07 -04:00
John Colvin 087d8ab8c9 Set text=auto in gitattributes
Otherwise git can cause files to show as "permanently modified" when you
apply changes from the index (e.g., by running 'git rebase', or 'git
stash pop')
2024-06-11 17:39:12 +00:00
John Colvin b69b7b3350 vscode: Use '\' dir separator in show_last_run.cmd
This script only runs on Windows
2024-06-08 15:08:25 -05:00
John Colvin 5620f33524 vscode: Add task to view logs for last PubSubServer run 2024-06-08 14:42:13 +00:00
John Colvin f3e33fa390 vscode: Add task to run PubSubServer without building 2024-06-07 16:41:42 +00:00
John Colvin 3b631e0d5f Fix a couple more typos in logs 2024-06-07 16:35:06 +00:00
2349 changed files with 39674 additions and 15952 deletions

No files matched your search

+11
View File
@@ -87,6 +87,15 @@ RUN dpkg -i /emumba/emumba-deribit-connector-3.3.1-Linux.LICENSE_DISABLED.deb \
# Pcap
RUN apt-get update && apt-get install -y libpcap0.8-dev
# Databento (deps)
RUN apt-get update && apt-get -y install libzstd-dev
# Docker in Docker
RUN apt-get update && apt-get install -y \
docker.io \
docker-compose \
docker-compose-v2
# Set up default executables (update-alternatives)
RUN \
update-alternatives --install /usr/bin/clang clang /usr/bin/clang-$clang_major 100; \
@@ -120,3 +129,5 @@ RUN \
-G sudo \
--password "$(openssl passwd -1 "$docker_pass")" \
"$docker_user"
RUN usermod -aG docker "$docker_user"
+4 -9
View File
@@ -18,17 +18,12 @@
"runArgs": [
"--shm-size",
"2gb",
"-p",
"8000:8000",
"-p",
"8001:8001",
"-p",
"8002:8002",
"-p",
"8003:8003"
"--network",
"host"
],
"mounts": [
"source=${localEnv:HOME}/.bts/devcontainer-mounts/${devcontainerId},target=/home/bts,type=bind"
"source=${localEnv:HOME}/.bts/devcontainer-mounts/${devcontainerId},target=/home/bts,type=bind",
"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
],
"customizations": {
"vscode": {
+30 -16
View File
@@ -207,17 +207,23 @@ dotnet_naming_symbols.parameters.applicable_accessibilities = public,internal,pr
dotnet_naming_symbols.private_symbols.applicable_kinds = property,method,field,event,delegate
dotnet_naming_symbols.private_symbols.applicable_accessibilities = protected,private
# local symbols
dotnet_naming_symbols.local_symbols.applicable_kinds = local
# parameter symbols
dotnet_naming_symbols.parameter_symbols.applicable_kinds = parameter
# Naming styles
dotnet_naming_style.snake_case.required_prefix =
dotnet_naming_style.snake_case.required_suffix =
dotnet_naming_style.snake_case.word_separator = _
dotnet_naming_style.snake_case.capitalization = all_lower
dotnet_naming_style.pascal_case.required_prefix =
dotnet_naming_style.pascal_case.required_suffix =
dotnet_naming_style.pascal_case.word_separator =
dotnet_naming_style.pascal_case.capitalization = pascal_case
dotnet_naming_style.private_snake_case.required_prefix = _
dotnet_naming_style.private_snake_case.required_suffix =
dotnet_naming_style.private_snake_case.word_separator = _
dotnet_naming_style.private_snake_case.capitalization = all_lower
dotnet_naming_style.camel_case.required_prefix =
dotnet_naming_style.camel_case.required_suffix =
dotnet_naming_style.camel_case.word_separator =
dotnet_naming_style.camel_case.capitalization = camel_case
dotnet_naming_style.user_types.required_prefix =
dotnet_naming_style.user_types.required_suffix =
@@ -229,6 +235,10 @@ dotnet_naming_style.interface.required_suffix =
dotnet_naming_style.interface.word_separator =
dotnet_naming_style.interface.capitalization = pascal_case
dotnet_naming_style.local.capitalization = camel_case
dotnet_naming_style.parameter.capitalization = camel_case
# rules
dotnet_naming_rule.user_types.symbols = user_types
dotnet_naming_rule.user_types.style = user_types
@@ -238,14 +248,18 @@ dotnet_naming_rule.interfaces.symbols = interface
dotnet_naming_rule.interfaces.style = interface_style
dotnet_naming_rule.interfaces.severity = suggestion
dotnet_naming_rule.public_members_snake_case.symbols = public_symbols
dotnet_naming_rule.public_members_snake_case.style = snake_case
dotnet_naming_rule.public_members_snake_case.severity = suggestion
dotnet_naming_rule.public_members.symbols = public_symbols
dotnet_naming_rule.public_members.style = pascal_case
dotnet_naming_rule.public_members.severity = suggestion
dotnet_naming_rule.private_members_snake_case.symbols = private_symbols
dotnet_naming_rule.private_members_snake_case.style = private_snake_case
dotnet_naming_rule.private_members_snake_case.severity = suggestion
dotnet_naming_rule.private_members.symbols = private_symbols
dotnet_naming_rule.private_members.style = camel_case
dotnet_naming_rule.private_members.severity = suggestion
dotnet_naming_rule.parameters.symbols = parameters
dotnet_naming_rule.parameters.style = snake_case
dotnet_naming_rule.variables_are_camel_case.severity = suggestion
dotnet_naming_rule.variables_are_camel_case.symbols = local_symbols
dotnet_naming_rule.variables_are_camel_case.style = local
dotnet_naming_rule.parameters.symbols = parameter_symbols
dotnet_naming_rule.parameters.style = parameter
dotnet_naming_rule.parameters.severity = suggestion
+4 -3
View File
@@ -1,3 +1,4 @@
*.cmd eol=crlf
*.cs eol=crlf
*.sh eol=lf
*.cmd text=auto eol=crlf
*.cs text=auto eol=crlf
*.sh text=auto eol=lf
*.py text=auto eol=lf
+1 -1
View File
@@ -1,6 +1,6 @@
[submodule "externals"]
path = externals
url = git@bitbucket.org:bluetradesys/externals.git
url = https://bitbucket.org/bluetradesys/externals.git
[submodule "vcpkg"]
path = vcpkg
url = https://github.com/microsoft/vcpkg.git
+195 -24
View File
@@ -1,6 +1,73 @@
{
"version": "0.2.0",
"configurations": [
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach"
},
{
"name": "Api Server",
"type": "coreclr",
"request": "launch",
"program": "",
"windows": {
"preLaunchTask": "Build Api Server (Debug)",
"program": "spark_api_server.exe",
"args": [
"run",
"--sparkd",
"http://127.0.0.1:8000/config"
],
"cwd": "build/Debug/bin/net7.0",
},
"stopAtEntry": false,
"console": "integratedTerminal"
},
{
"name": "Api Server (Skip Build)",
"type": "coreclr",
"request": "launch",
"program": "",
"windows": {
"program": "spark_api_server.exe",
"args": [
"run",
"--sparkd",
"http://127.0.0.1:8000/config"
],
"cwd": "build/Debug/bin/net7.0",
},
"stopAtEntry": false,
"console": "integratedTerminal"
},
{
"name": "Bts.DataStreamWebApp.Server",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "Build DataStream WebApp (Debug)",
"program": "${workspaceFolder}/build/Debug/bin/net7.0/Bts.DataStreamWebApp.Server.dll",
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"env": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "${input:dataStreamWebAppServerEnvironment}",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
{
"name": "Bts.DataStreamWebApp.Server (Skip Build)",
"type": "coreclr",
"request": "launch",
"program": "${workspaceFolder}/build/Debug/bin/net7.0/Bts.DataStreamWebApp.Server.dll",
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"env": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "${input:dataStreamWebAppServerEnvironment}",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
{
"name": "DataStreamPublisher",
"type": "coreclr",
@@ -13,11 +80,28 @@
"windows": {
"program": "DataStreamPublisher.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/DataStreamPublisher/Configurations/${input:dataStreamPublisherConfig}.json"
],
"cwd": "build/Debug/bin/net7.0",
"env": {
"DOTNET_ENVIRONMENT": "${input:dataStreamPublisherEnvironment}",
},
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"stopAtEntry": false,
"console": "integratedTerminal"
},
{
"name": "DataStreamPublisher (Skip Build)",
"type": "coreclr",
"request": "launch",
"program": "",
"linux": {
"program": "DataStreamPublisher",
},
"windows": {
"program": "DataStreamPublisher.exe",
},
"env": {
"DOTNET_ENVIRONMENT": "${input:dataStreamPublisherEnvironment}",
},
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"stopAtEntry": false,
"console": "integratedTerminal"
},
@@ -25,7 +109,7 @@
"name": "DataStreamQueryAnalyzer",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "Build DataStream (Debug)",
"preLaunchTask": "Build DataStreamQueryAnalyzer (Debug)",
"program": "",
"windows": {
"program": "DataStreamQueryAnalyzer.exe",
@@ -53,11 +137,28 @@
"windows": {
"program": "PubSubServer.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json"
],
"cwd": "build/Debug/bin/net7.0",
"env": {
"DOTNET_ENVIRONMENT": "${input:pubSubServerEnvironment}",
},
"stopAtEntry": false,
"console": "integratedTerminal"
},
{
"name": "PubSubServer (Skip Build)",
"type": "coreclr",
"request": "launch",
"program": "",
"linux": {
"program": "PubSubServer",
},
"windows": {
"program": "PubSubServer.exe",
},
"cwd": "build/Debug/bin/net7.0",
"env": {
"DOTNET_ENVIRONMENT": "${input:pubSubServerEnvironment}",
},
"stopAtEntry": false,
"console": "integratedTerminal"
},
@@ -69,6 +170,10 @@
"windows": {
"preLaunchTask": "Build Spark (Debug)",
"program": "Spark.exe",
"args": [
"--sparkd",
"http://127.0.0.1:8000/config"
],
"cwd": "build/Debug/bin/net7.0-windows",
},
"stopAtEntry": false,
@@ -93,43 +198,99 @@
},
{
"name": "Sparkd",
"type": "coreclr",
"type": "cppdbg",
"request": "launch",
"preLaunchTask": "Build Sparkd (Debug | Clang)",
"program": "",
"linux": {
"program": "sparkd",
"program": "${workspaceFolder}/build/sparkd-debug-clang/src/app_sparkd/sparkd",
},
"args": [
"run"
],
"cwd": "build/sparkd-debug-clang/server",
"cwd": "${workspaceFolder}/build/sparkd-debug-clang/src/app_sparkd",
"stopAtEntry": false,
"console": "integratedTerminal"
}
"externalConsole": false,
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
},
{
"description": "Set Disassembly Flavor to Intel",
"text": "-gdb-set disassembly-flavor intel",
"ignoreFailures": true
}
]
},
{
"name": "Sparkd (Skip Build)",
"type": "cppdbg",
"request": "launch",
"program": "",
"linux": {
"program": "${workspaceFolder}/build/sparkd-debug-clang/src/app_sparkd/sparkd",
},
"args": [
"test-py-md-provider"
],
"cwd": "${workspaceFolder}/build/sparkd-debug-clang/src/app_sparkd",
"stopAtEntry": false,
"externalConsole": false,
"MIMode": "gdb",
"setupCommands": [
{
"description": "Enable pretty-printing for gdb",
"text": "-enable-pretty-printing",
"ignoreFailures": true
},
{
"description": "Set Disassembly Flavor to Intel",
"text": "-gdb-set disassembly-flavor intel",
"ignoreFailures": true
}
]
},
],
"inputs": [
{
"id": "dataStreamPublisherConfig",
"id": "dataStreamPublisherEnvironment",
"type": "pickString",
"description": "Publisher Configuration",
"options": [
"MarketDataPublisher",
"SimpleEchoClient",
"SimpleTestDataPublisher",
"TestDataPublisher",
"Development",
"MarketData",
"Production",
"ProductionAws",
"Staging",
],
"default": "MarketDataPublisher",
},
{
"id": "pubSubServerConfig",
"id": "pubSubServerEnvironment",
"type": "pickString",
"description": "PubSub Configuration",
"options": [
"PubSubServer",
"SimpleEchoClient",
"Development",
"EchoServer",
"Production",
"ProductionAws",
"Staging",
],
"default": "PubSubServer",
"default": "Development",
},
{
"id": "dataStreamWebAppServerEnvironment",
"type": "pickString",
"description": "DataStream WebApp Environment",
"options": [
"Development",
"Production",
"ProductionAws",
"Staging",
]
},
{
"id": "sparkConfigServer",
@@ -137,5 +298,15 @@
"description": "Config Server URL",
"default": "http://localhost:8000/config",
},
{
"id": "isSparkApiEnabled",
"type": "pickString",
"description": "Enable Spark Api (connect to sparkd)",
"options": [
"True",
"False",
],
"default": "True",
}
]
}
+287 -3
View File
@@ -121,6 +121,117 @@
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStreamQueryAnalyzer",
"group": "build",
"type": "process",
"command": "",
"windows": {
"command": "tools/build_data_stream_query_analyzer.cmd",
},
"args": [
"--configuration",
"${input:buildConfiguration}",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStreamQueryAnalyzer (Debug)",
"group": "build",
"type": "process",
"command": "",
"windows": {
"command": "tools/build_data_stream_query_analyzer.cmd",
},
"args": [
"--configuration",
"Debug",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStreamQueryAnalyzer (Release)",
"group": "build",
"type": "process",
"command": "",
"windows": {
"command": "tools/build_data_stream_query_analyzer.cmd",
},
"args": [
"--configuration",
"Release",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStream WebApp",
"group": "build",
"type": "process",
"command": "",
"linux": {
"command": "tools/build_data_stream_web_app.sh",
},
"windows": {
"command": "tools/build_data_stream_web_app.cmd",
},
"args": [
"--configuration",
"${input:buildConfiguration}",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStream WebApp (Debug)",
"group": "build",
"type": "process",
"command": "",
"linux": {
"command": "tools/build_data_stream_web_app.sh",
},
"windows": {
"command": "tools/build_data_stream_web_app.cmd",
},
"args": [
"--configuration",
"Debug",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build DataStream WebApp (Release)",
"group": "build",
"type": "process",
"command": "",
"linux": {
"command": "tools/build_data_stream_web_app.sh",
},
"windows": {
"command": "tools/build_data_stream_web_app.cmd",
},
"args": [
"--configuration",
"Release",
],
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Build Spark",
"group": "build",
@@ -303,6 +414,22 @@
"color": "terminal.ansiCyan"
},
},
{
"label": "Publish DataStreamWebApp",
"group": "build",
"type": "process",
"command": "",
"linux": {
"command": "tools/publish_data_stream_web_app.sh",
},
"windows": {
"command": "tools/publish_data_stream_web_app.cmd",
},
"icon": {
"id": "terminal",
"color": "terminal.ansiCyan"
},
},
{
"label": "Edit DataStream Config",
"type": "process",
@@ -353,7 +480,7 @@
},
{
"label": "Run DataStreamQueryAnalyzer",
"dependsOn": "Build DataStream",
"dependsOn": "Build DataStreamQueryAnalyzer",
"type": "process",
"command": "",
"windows": {
@@ -373,7 +500,7 @@
},
{
"label": "Run DataStreamQueryAnalyzer (Debug)",
"dependsOn": "Build DataStream (Debug)",
"dependsOn": "Build DataStreamQueryAnalyzer (Debug)",
"type": "process",
"command": "",
"windows": {
@@ -393,7 +520,7 @@
},
{
"label": "Run DataStreamQueryAnalyzer (Release)",
"dependsOn": "Build DataStream (Release)",
"dependsOn": "Build DataStreamQueryAnalyzer (Release)",
"type": "process",
"command": "",
"windows": {
@@ -480,6 +607,89 @@
"color": "terminal.ansiGreen",
},
},
{
"label": "Run DataStream WebApp (Debug)",
"dependsOn": "Build DataStream WebApp (Debug)",
"command": "",
"linux": {
"command": "Bts.DataStreamWebApp.Server",
},
"windows": {
"command": "Bts.DataStreamWebApp.Server.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/Bts.DataStreamWebApp.Server/Configurations/DevServer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"env": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
"icon": {
"id": "run",
"color": "terminal.ansiGreen",
},
},
{
"label": "Run DataStream WebApp (Release)",
"dependsOn": "Build DataStream WebApp (Release)",
"command": "",
"linux": {
"command": "Bts.DataStreamWebApp.Server",
},
"windows": {
"command": "Bts.DataStreamWebApp.Server.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/Bts.DataStreamWebApp.Server/Configurations/DevServer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0",
"env": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "Production",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
"icon": {
"id": "run",
"color": "terminal.ansiGreen",
},
},
{
"label": "Run DataStream WebApp (Release | Skip Build)",
"command": "",
"linux": {
"command": "Bts.DataStreamWebApp.Server",
},
"windows": {
"command": "Bts.DataStreamWebApp.Server.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/Bts.DataStreamWebApp.Server/Configurations/DevServer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0",
"env": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "Production",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
"icon": {
"id": "run",
"color": "terminal.ansiGreen",
},
},
{
"label": "Run PubSubServer",
"dependsOn": "Build DataStream",
@@ -526,6 +736,28 @@
"color": "terminal.ansiGreen",
},
},
{
"label": "Run PubSubServer (Debug | Skip Build)",
"type": "process",
"command": "",
"linux": {
"command": "PubSubServer",
},
"windows": {
"command": "PubSubServer.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0"
},
"icon": {
"id": "run",
"color": "terminal.ansiGreen",
},
},
{
"label": "Run PubSubServer (Release)",
"dependsOn": "Build DataStream (Release)",
@@ -549,6 +781,28 @@
"color": "terminal.ansiGreen",
},
},
{
"label": "Run PubSubServer (Release | Skip Build)",
"type": "process",
"command": "",
"linux": {
"command": "PubSubServer",
},
"windows": {
"command": "PubSubServer.exe",
},
"args": [
"--config",
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0"
},
"icon": {
"id": "run",
"color": "terminal.ansiGreen",
},
},
{
"label": "Run Spark",
"dependsOn": "Build Spark",
@@ -684,6 +938,36 @@
"id": "wrench"
},
},
{
"label": "View PubSubServer Logs (Last Run)",
"type": "process",
"command": "",
"linux": {
"command": "tools/logging/show_last_run.sh",
"args": [
"${env:HOME}/.local/share/DataStream/data/logging.PubSubServer",
"Starting PubSubServer"
]
},
"windows": {
"command": "tools/logging/show_last_run.cmd",
"args": [
"DataStream",
"PubSubServer.exe",
"Starting PubSubServer"
]
},
"presentation": {
"reveal": "always",
"focus": true,
"panel": "shared",
"showReuseMessage": false,
"close": true,
},
"icon": {
"id": "wrench"
},
},
],
"inputs": [
{
+15 -46
View File
@@ -18,10 +18,10 @@ set(
# SHARED_MEMORY_PREFIX serves as a kind of namespace so multiple applications
# can run simultaneously without fighting over the same shm files (e.g., sparkd,
# api server, etc.).
set(SHARED_MEMORY_PREFIX "bts" CACHE STRING "Prefix to use when creating shared memory files")
set(SPARK_BIN "${EXTERNALS_ROOT}/bin")
set(SPARK_INCLUDE "${EXTERNALS_ROOT}/include")
set(SPARK_LIB "${EXTERNALS_ROOT}/lib/$<CONFIG>")
set(SHARED_MEMORY_PREFIX "\${BTS_SHARED_MEMORY_PREFIX}" CACHE STRING "Prefix to use when creating shared memory files")
set(EXTERNALS_BIN "${EXTERNALS_ROOT}/bin")
set(EXTERNALS_INCLUDE "${EXTERNALS_ROOT}/include")
set(EXTERNALS_LIB "${EXTERNALS_ROOT}/lib/$<CONFIG>")
set(BUILD_DOCS OFF CACHE BOOL "Build documentation")
set(BUILD_SDK OFF CACHE BOOL "Build the sdk")
set(EXPORT_MD_CONSUMER OFF CACHE BOOL "Export (extern) C functions for MD consumer interop")
@@ -33,25 +33,18 @@ else()
set(BUILD_ADMIN_WEB_APP ON CACHE BOOL "Build the Admin WebApp")
endif()
include_directories(BEFORE SYSTEM ${SPARK_INCLUDE} ${EXTERNALS_ROOT}/include/zlib)
link_directories(${SPARK_LIB})
include_directories(BEFORE SYSTEM ${EXTERNALS_INCLUDE} ${EXTERNALS_ROOT}/include/zlib)
link_directories(${EXTERNALS_LIB})
add_compile_definitions(SHARED_MEMORY_PREFIX="${SHARED_MEMORY_PREFIX}")
add_compile_definitions(APP_DATA_DIR="${APP_DATA_DIR}")
if(DEFINED APP_DATA_DIR_OVERRIDE)
add_compile_definitions(APP_DATA_DIR_OVERRIDE="${APP_DATA_DIR_OVERRIDE}")
if(DEFINED APP_DATA_DIR)
add_compile_definitions(APP_DATA_DIR="${APP_DATA_DIR}")
endif()
add_compile_definitions(EXPORT_MD_CONSUMER="${EXPORT_MD_CONSUMER}")
if(WIN32)
# subdirs
add_subdirectory(src/lib)
add_subdirectory(src/lib_spark)
else()
# make sparkd look for libspark.so in its directory
set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE)
set(CMAKE_INSTALL_RPATH ".")
set(CMAKE_INSTALL_PREFIX "${CMAKE_SOURCE_DIR}/install")
set(SPARK_INSTALL_PREFIX "${CMAKE_INSTALL_PREFIX}/$<CONFIG>")
if(BUILD_SPARKD)
# Note: All macros that begin w/ 'CCAPI' are ccapi's macros (defined and/or
# read by their code). We override some below.
@@ -111,10 +104,14 @@ else()
# subdirs
if (BUILD_SPARKD)
add_subdirectory(src/server/sparkd)
add_subdirectory(src/lib_system)
add_subdirectory(src/lib_domain)
add_subdirectory(src/lib_python_md)
add_subdirectory(src/lib_logging)
add_subdirectory(src/app_sparkd)
add_subdirectory(src/data_files)
endif()
add_subdirectory(src/lib)
add_subdirectory(src/lib_spark)
if (BUILD_ADMIN_WEB_APP)
add_subdirectory(src/Spark.Admin.WebApp)
endif()
@@ -124,32 +121,4 @@ else()
if(BUILD_SDK)
add_subdirectory(src/sdk)
endif()
if(BUILD_SPARKD)
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/server
COMMAND mkdir -p ${CMAKE_BINARY_DIR}/server
COMMAND ln ARGS -sf ${CMAKE_BINARY_DIR}/src/server/sparkd/sparkd ${CMAKE_BINARY_DIR}/server
COMMAND ln ARGS -sf ${CMAKE_BINARY_DIR}/data_files ${CMAKE_BINARY_DIR}/server
COMMAND ln ARGS -sf ${CMAKE_BINARY_DIR}/src/lib/libspark.so ${CMAKE_BINARY_DIR}/server
COMMAND
ln ARGS -sf
${EXTERNALS_ROOT}/lib/Release/libOnixS.IceImpactMulticastPriceFeedHandlerCpp.so.3.16.0.0
${CMAKE_BINARY_DIR}/server
COMMAND ln ARGS -sf ${EXTERNALS_ROOT}/lib/Release/libACE-5.7.5.so ${CMAKE_BINARY_DIR}/server
COMMAND mkdir ARGS -p ${CMAKE_BINARY_DIR}/server/plugins
COMMENT "Creating server runtime layout"
DEPENDS sparkd data_files_dir)
add_custom_target(server_layout ALL DEPENDS ${CMAKE_BINARY_DIR}/server)
install(
DIRECTORY "${CMAKE_BINARY_DIR}/server/plugins"
DESTINATION "${SPARK_INSTALL_PREFIX}/bin")
install(
FILES
"${SPARK_LIB}/libACE-5.7.5.so"
"${SPARK_LIB}/libOnixS.IceImpactMulticastPriceFeedHandlerCpp.so.3.16.0.0"
DESTINATION "${SPARK_INSTALL_PREFIX}/bin")
endif()
endif()
+24 -43
View File
@@ -20,8 +20,6 @@
"SPARK_CMAKE_CXX_FLAGS": "/MP /WX /wd4250 /wd4251 /wd4244 /wd4267 /wd4355 /wd4275 /wd4345 /bigobj /EHsc /DCURL_STATICLIB /D_WINSOCK_DEPRECATED_NO_WARNINGS /D__TBB_NO_IMPLICIT_LINKAGE /DBOOST_ALL_NO_LIB /std:c++20 /D_SCL_SECURE_NO_WARNINGS /D_CRT_SECURE_NO_WARNINGS /D_WIN32_WINNT=0x0600"
},
"cacheVariables": {
"APP_DATA_DIR": "${dollar}{APPDATA}/bts",
"APP_DATA_DIR_OVERRIDE": "${dollar}{SPARK_DATA_DIR}",
"CMAKE_CXX_FLAGS": "$env{SPARK_CMAKE_CXX_FLAGS}",
"CMAKE_CXX_FLAGS_DEBUG": "$env{SPARK_CMAKE_CXX_FLAGS} /MTd",
"CMAKE_CXX_FLAGS_RELEASE": "$env{SPARK_CMAKE_CXX_FLAGS} /MT",
@@ -31,13 +29,11 @@
"CMAKE_TOOLCHAIN_FILE": "${sourceDir}/vcpkg/scripts/buildsystems/vcpkg.cmake",
"EXTERNALS_ROOT": "${sourceDir}/externals/windows",
"CONFIG_DEFAULTS_PATH": "config/profiles/default/*.conf",
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0",
"DOTNET_WINDOWS_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0-windows",
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0;${sourceDir}/build/$<CONFIG>/bin/net7.0-windows",
"EXPORT_MD_CONSUMER": {
"type": "BOOL",
"value": "ON"
},
"SHARED_MEMORY_PREFIX": "spark"
}
}
},
@@ -53,15 +49,9 @@
"binaryDir": "${sourceDir}/build/${presetName}",
"cacheVariables": {
"platform": "",
"APP_DATA_DIR": "${dollar}{HOME}/sparkdata",
"APP_DATA_DIR_OVERRIDE": "${dollar}{SPARKD_DATA_DIR}",
"CMAKE_BUILD_WITH_INSTALL_RPATH": {
"type": "BOOL",
"value": "ON"
},
"CMAKE_CXX_STANDARD": "20",
"CMAKE_INSTALL_RPATH": ".",
"CMAKE_INSTALL_PREFIX": "${sourceDir}/install/${presetName}/sparkd",
"CMAKE_INSTALL_PREFIX": "${sourceDir}/install/${presetName}",
"CMAKE_PREFIX_PATH": "${sourceDir}/cmake/;${sourceDir}/.venv/lib/python3.10/site-packages/pybind11/share/cmake/pybind11",
"CMAKE_EXE_LINKER_FLAGS": "-Wno-undef",
"EXTERNALS_ROOT": "${sourceDir}/externals/linux",
"BUILD_SDK": {
@@ -71,8 +61,7 @@
"BUILD_ADMIN_WEB_APP": {
"type": "BOOL",
"value": "ON"
},
"SHARED_MEMORY_PREFIX": "sparkd"
}
}
},
@@ -153,11 +142,7 @@
{
"name": "spark-api-server-windows",
"inherits": "spark",
"displayName": "Spark Api Server (Windows)",
"cacheVariables": {
"APP_DATA_DIR_OVERRIDE": "${dollar}{SPARK_API_SERVER_DATA_DIR}",
"SHARED_MEMORY_PREFIX": "spark_api_server"
}
"displayName": "Spark Api Server (Windows)"
},
{
@@ -166,7 +151,6 @@
"displayName": "Spark Api Server (Linux)",
"generator": "Ninja Multi-Config",
"cacheVariables": {
"APP_DATA_DIR_OVERRIDE": "${dollar}{SPARK_API_SERVER_DATA_DIR}",
"BUILD_ADMIN_WEB_APP": {
"type": "BOOL",
"value": "OFF"
@@ -179,9 +163,12 @@
"CMAKE_CXX_COMPILER": "clang++",
"CMAKE_CXX_FLAGS_DEBUG": "-g -fPIC -Wall -pedantic -Wextra -Werror -Wno-missing-braces -Wno-ignored-qualifiers --std=c++20 -fdiagnostics-color -DDEBUG",
"CMAKE_CXX_FLAGS_RELEASE": "-fPIC -Wall -pedantic -Wextra -Werror -Wno-missing-braces -Wno-ignored-qualifiers --std=c++20 -fdiagnostics-color -fno-omit-frame-pointer -DNDEBUG -O3",
"CMAKE_INSTALL_PREFIX": "${sourceDir}/install/${presetName}",
"CMAKE_EXPORT_COMPILE_COMMANDS": {
"type": "BOOL",
"value": "ON"
},
"CONFIG_DEFAULTS_PATH": "config/profiles/default/*.conf",
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0",
"DOTNET_BINARY_DIR": "${sourceDir}/build/Spark.Api.Server/$<CONFIG>/bin/net7.0",
"EXPORT_MD_CONSUMER": {
"type": "BOOL",
"value": "ON"
@@ -219,10 +206,8 @@
"description": "Data Stream utils ported over from Edge",
"binaryDir": "${sourceDir}/build/${presetName}",
"cacheVariables": {
"APP_DATA_DIR": "${dollar}{APPDATA}/bts/DataStream",
"APP_DATA_DIR_OVERRIDE": "${dollar}{DATA_STREAM_DATA_DIR}",
"CONFIG_DEFAULTS_PATH": "config/profiles/default/*.conf;config/profiles/data_stream/*.conf",
"SHARED_MEMORY_PREFIX": "data_stream"
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0;${sourceDir}/build/$<CONFIG>/bin/net7.0-windows"
}
},
@@ -238,8 +223,6 @@
"inherits": "sparkd",
"generator": "Ninja Multi-Config",
"cacheVariables": {
"APP_DATA_DIR": "${dollar}{HOME}/.local/share/DataStream",
"APP_DATA_DIR_OVERRIDE": "${dollar}{DATA_STREAM_DATA_DIR}",
"BUILD_ADMIN_WEB_APP": {
"type": "BOOL",
"value": "OFF"
@@ -256,14 +239,13 @@
"CMAKE_CXX_COMPILER": "clang++",
"CMAKE_CXX_FLAGS_DEBUG": "-g -fPIC -Wall -pedantic -Wextra -Werror -Wno-missing-braces -Wno-ignored-qualifiers --std=c++20 -fdiagnostics-color -DDEBUG",
"CMAKE_CXX_FLAGS_RELEASE": "-fPIC -Wall -pedantic -Wextra -Werror -Wno-missing-braces -Wno-ignored-qualifiers --std=c++20 -fdiagnostics-color -fno-omit-frame-pointer -DNDEBUG -O3",
"CMAKE_INSTALL_PREFIX": "${sourceDir}/install/${presetName}",
"CONFIG_DEFAULTS_PATH": "config/profiles/default/*.conf;config/profiles/data_stream/*.conf",
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0",
"CONFIG_PROFILES": "default;data_stream",
"DOTNET_BINARY_DIR": "${sourceDir}/build/$<CONFIG>/bin/net7.0;${sourceDir}/build/$<CONFIG>/bin/net8.0",
"EXPORT_MD_CONSUMER": {
"type": "BOOL",
"value": "ON"
},
"SHARED_MEMORY_PREFIX": "data_stream"
}
}
}
],
@@ -302,16 +284,7 @@
"lhs": "${hostSystemName}",
"rhs": "Linux"
},
"hidden": true,
"targets": [
"data_files_dir",
"admin_web_app_bundle",
"install",
"libspark",
"rapidjson_project",
"server_layout",
"sparkd"
]
"hidden": true
},
{
@@ -321,6 +294,14 @@
"inherits": "sparkd"
},
{
"name": "sparkd-debug-clang-j6",
"configurePreset": "sparkd-debug-clang",
"displayName": "Sparkd (Debug | Clang | jobs=6)",
"inherits": "sparkd",
"jobs": 6
},
{
"name": "sparkd-debug-gcc",
"configurePreset": "sparkd-debug-gcc",
+5
View File
@@ -0,0 +1,5 @@
macro(export_to_parent)
foreach(var IN ITEMS ${ARGN})
set(${var} ${${var}} PARENT_SCOPE)
endforeach()
endmacro()
+66
View File
@@ -0,0 +1,66 @@
include_guard(DIRECTORY)
include(${CMAKE_SOURCE_DIR}/cmake/ExportToParent.cmake)
include(${CMAKE_SOURCE_DIR}/cmake/TernaryOperator.cmake)
function(init_library)
set(_options SHARED STATIC)
set(_one_value_args SOURCE_DIR)
set(_multi_value_args)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
if (_ARGS_SHARED AND _ARGS_STATIC)
message(FATAL_ERROR "Invalid args passed to init_library: cannot specify STATIC and SHARED")
endif()
set_ternary(_source_dir _ARGS_SOURCE_DIR _ARGS_SOURCE_DIR ${PROJECT_NAME})
# Glob cpp sources
file(
GLOB_RECURSE ${PROJECT_NAME}_SOURCES
RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
${_source_dir}/*.cpp
)
# Note: cpp files inside 'tests' dir(s) are considered tests.
# Remove tests from main sources, and add them to test sources
foreach(item ${${PROJECT_NAME}_SOURCES})
if(${item} MATCHES "tests/*.cpp")
list(REMOVE_ITEM ${PROJECT_NAME}_SOURCES ${item})
list(APPEND ${PROJECT_NAME}_TEST_SOURCES ${item})
endif()
endforeach()
export_to_parent(
${PROJECT_NAME}_SOURCES
${PROJECT_NAME}_TEST_SOURCES
)
# Add main library
if (_ARGS_STATIC)
add_library(${PROJECT_NAME} STATIC ${${PROJECT_NAME}_SOURCES})
elseif(_ARGS_SHARED)
add_library(${PROJECT_NAME} SHARED ${${PROJECT_NAME}_SOURCES})
else()
add_library(${PROJECT_NAME} ${${PROJECT_NAME}_SOURCES})
endif()
target_include_directories(
${PROJECT_NAME}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_link_directories(
${PROJECT_NAME}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_LIBDIR}>
)
# Add tests executable (if there are any)
if (${PROJECT_NAME}_TEST_SOURCES)
add_executable(${PROJECT_NAME}-tests ${${PROJECT_NAME}_TEST_SOURCES})
target_link_libraries(${PROJECT_NAME}-tests PRIVATE ${PROJECT_NAME})
# Catch2
find_package(Catch2 CONFIG REQUIRED COMPONENTS Catch2WithMain)
target_link_libraries(${PROJECT_NAME}-tests PRIVATE Catch2::Catch2WithMain)
endif()
endfunction()
+55
View File
@@ -0,0 +1,55 @@
function(init_shared_library)
include(${CMAKE_SOURCE_DIR}/cmake/ExportToParent.cmake)
set(source_dir)
if (${ARGC} EQUAL 0)
set(source_dir ${PROJECT_NAME})
elseif(${ARGC} GREATER 1)
message(FATAL_ERROR "init_shared_library received invalid arguments. Usage: init_shared_library(<OPTIONAL:source_dir>)")
else()
set(source_dir ${ARGV})
endif()
# Glob cpp sources
file(
GLOB_RECURSE ${PROJECT_NAME}_SOURCES
RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
${source_dir}/*.cpp
)
# Note: cpp files inside 'tests' dir(s) are considered tests.
# Remove tests from main sources, and add them to test sources
foreach(item ${${PROJECT_NAME}_SOURCES})
if(${item} MATCHES "tests/*.cpp")
list(REMOVE_ITEM ${PROJECT_NAME}_SOURCES ${item})
list(APPEND ${PROJECT_NAME}_TEST_SOURCES ${item})
endif()
endforeach()
export_to_parent(
${PROJECT_NAME}_SOURCES
${PROJECT_NAME}_TEST_SOURCES
)
# Add main library
add_library(${PROJECT_NAME} SHARED ${${PROJECT_NAME}_SOURCES})
target_include_directories(
${PROJECT_NAME}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>
)
target_link_directories(
${PROJECT_NAME}
PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_LIBDIR}>
)
# Add tests executable (if there are any)
if (${PROJECT_NAME}_TEST_SOURCES)
add_executable(${PROJECT_NAME}-tests ${${PROJECT_NAME}_TEST_SOURCES})
target_link_libraries(${PROJECT_NAME}-tests PRIVATE ${PROJECT_NAME})
# Catch2
find_package(Catch2 CONFIG REQUIRED COMPONENTS Catch2WithMain)
target_link_libraries(${PROJECT_NAME}-tests PRIVATE Catch2::Catch2WithMain)
endif()
endfunction(init_shared_library)
+30
View File
@@ -0,0 +1,30 @@
include(${CMAKE_SOURCE_DIR}/cmake/PackBasic.cmake)
include(GNUInstallDirs)
function(install_basic)
file(
GLOB_RECURSE ${PROJECT_NAME}_HEADERS
RELATIVE ${CMAKE_CURRENT_SOURCE_DIR}
*.h)
target_sources(${PROJECT_NAME}
PUBLIC
FILE_SET HEADERS FILES ${${PROJECT_NAME}_HEADERS})
install(
TARGETS ${PROJECT_NAME}
EXPORT ${PROJECT_NAME}Targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
FILE_SET HEADERS DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
)
install(
EXPORT ${PROJECT_NAME}Targets
FILE ${PROJECT_NAME}Targets.cmake
NAMESPACE ${PROJECT_NAME}::
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME})
pack_basic()
endfunction()
+36
View File
@@ -0,0 +1,36 @@
include(CMakePackageConfigHelpers)
function(pack_basic)
find_package(Git QUIET)
if(GIT_FOUND)
execute_process(
COMMAND ${GIT_EXECUTABLE} describe --tags --always --dirty
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
OUTPUT_VARIABLE PROJECT_VERSION
OUTPUT_STRIP_TRAILING_WHITESPACE
)
endif()
if(NOT GIT_FOUND OR PROJECT_VERSION STREQUAL "")
set(PROJECT_VERSION "0.0.0")
endif()
write_basic_package_version_file(
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
VERSION ${PROJECT_VERSION}
COMPATIBILITY AnyNewerVersion
)
configure_package_config_file(
"${CMAKE_CURRENT_SOURCE_DIR}/Config.cmake.in"
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}
)
install(
FILES
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Config.cmake"
"${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}ConfigVersion.cmake"
DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME})
endfunction()
-145
View File
@@ -1,145 +0,0 @@
set(Protobuf_PROTOC_EXECUTABLE "${SPARK_BIN}/protoc")
set(PROTOBUF_INCLUDE_DIRS "${SPARK_INCLUDE}")
set(PROTOBUF_STATIC_LIB "libprotobuf")
function(Protobuf_Generate_Cpp SRCS HDRS)
set(${SRCS})
set(${HDRS})
foreach(FIL ${ARGN})
get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
get_filename_component(FIL_WE ${FIL} NAME_WE)
get_filename_component(FIL_DIR ${FIL} DIRECTORY)
if(FIL_DIR)
set(FIL_WE "${FIL_DIR}/${FIL_WE}")
endif()
list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc")
list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h")
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h"
COMMAND
${Protobuf_PROTOC_EXECUTABLE} ARGS --cpp_out=${CMAKE_BINARY_DIR}/src -I
${PROTOBUF_INCLUDE_DIRS} -I ${CMAKE_SOURCE_DIR}/src ${ABS_FIL}
DEPENDS ${ABS_FIL}
COMMENT "Running C++ protocol buffer compiler on ${FIL}"
VERBATIM)
endforeach()
set_source_files_properties(${${SRCS}} ${${HDRS}} PROPERTIES GENERATED TRUE)
# remove some warnings from protobuf generated files
if(WIN32)
set_source_files_properties(${${SRCS}} PROPERTIES COMPILE_FLAGS
"/wd4309 /wd4838")
else()
set_source_files_properties(
${${SRCS}}
PROPERTIES
COMPILE_FLAGS
"-Wno-unused-parameter -Wno-effc++ -Wno-pedantic -Wno-sign-compare -Wno-uninitialized"
)
endif()
set(${SRCS}
${${SRCS}}
PARENT_SCOPE)
set(${HDRS}
${${HDRS}}
PARENT_SCOPE)
endfunction()
function(Service_Compiler_Generate_Cpp SRCS HDRS)
set(${SRCS})
set(${HDRS})
foreach(FIL ${ARGN})
get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
get_filename_component(FIL_WE ${FIL} NAME_WE)
get_filename_component(FIL_DIR ${FIL} DIRECTORY)
if(FIL_DIR)
set(FIL_WE "${FIL_DIR}/${FIL_WE}")
endif()
list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc")
list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h")
list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.cc")
list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.h")
list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.cc")
list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.h")
set(TEMPLATES
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/templates/cpp/client.pb.cc
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/templates/cpp/client.pb.h
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/templates/cpp/server.pb.cc
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/templates/cpp/server.pb.h
)
if(WIN32)
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.h"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.h"
COMMAND SET PYTHONPATH=${SPARK_PYTHON}
COMMAND ${CMAKE_SOURCE_DIR}/.venv/Scripts/activate.bat
COMMAND
${Protobuf_PROTOC_EXECUTABLE} ARGS --cpp_out=${CMAKE_BINARY_DIR}/src
--plugin=protoc-gen-service=${CMAKE_SOURCE_DIR}/src/lib/api/service_compiler/plugin.cmd
--service_out=lang=cpp:${CMAKE_BINARY_DIR}/src -I
${PROTOBUF_INCLUDE_DIRS} -I ${CMAKE_SOURCE_DIR}/src ${ABS_FIL}
DEPENDS ${ABS_FIL} ${TEMPLATES}
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/plugin.py
COMMENT "Running C++ service compiler on ${FIL}"
VERBATIM)
else()
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.client.pb.h"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.cc"
"${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.server.pb.h"
COMMAND ${CMAKE_SOURCE_DIR}/tools/venv_exec.sh
ARGS
${Protobuf_PROTOC_EXECUTABLE}
--cpp_out=${CMAKE_BINARY_DIR}/src
--plugin=protoc-gen-service=${CMAKE_SOURCE_DIR}/src/lib/api/service_compiler/plugin.py
--service_out=lang=cpp,make_file_name=${ABS_FIL}:${CMAKE_BINARY_DIR}/src
-I ${PROTOBUF_INCLUDE_DIRS} -I ${CMAKE_SOURCE_DIR}/src ${ABS_FIL}
DEPENDS ${ABS_FIL} ${TEMPLATES}
${CMAKE_CURRENT_SOURCE_DIR}/api/service_compiler/plugin.py
COMMENT "Running C++ service compiler on ${FIL}"
VERBATIM)
endif()
endforeach()
# remove some warnings from protobuf generated files
if(WIN32)
set_source_files_properties(${${SRCS}} PROPERTIES COMPILE_FLAGS
"/wd4309 /wd4838")
else()
set_source_files_properties(
${${SRCS}}
PROPERTIES
COMPILE_FLAGS
"-Wno-unused-parameter -Wno-effc++ -Wno-pedantic -Wno-sign-compare -Wno-uninitialized"
)
endif()
set(${SRCS}
${${SRCS}}
PARENT_SCOPE)
set(${HDRS}
${${HDRS}}
PARENT_SCOPE)
endfunction()
+67
View File
@@ -0,0 +1,67 @@
include_guard(GLOBAL)
function(protobuf_generate_cpp)
set(_options)
set(_one_value_args CXX_SOURCES CXX_HEADERS CXX_OUT_DIR PROTOC)
set(_multi_value_args PROTO_INCLUDE_DIRS PROTO_SOURCES)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
if (NOT DEFINED _ARGS_PROTOC)
set(_ARGS_PROTOC "${EXTERNALS_BIN}/protoc")
endif()
set(${_ARGS_CXX_SOURCES})
set(${_ARGS_CXX_HEADERS})
set(_proto_include_cli_args)
foreach(_dir ${_ARGS_PROTO_INCLUDE_DIRS})
list(APPEND _proto_include_cli_args -I ${_dir})
endforeach()
foreach(_fil ${_ARGS_PROTO_SOURCES})
get_filename_component(_abs_fil ${_fil} ABSOLUTE)
get_filename_component(_fil_we ${_fil} NAME_WE)
get_filename_component(_fil_dir ${_fil} DIRECTORY)
if(_fil_dir)
set(_fil_we "${_fil_dir}/${_fil_we}")
endif()
list(APPEND ${_ARGS_CXX_SOURCES} "${_ARGS_CXX_OUT_DIR}/${_fil_we}.pb.cc")
list(APPEND ${_ARGS_CXX_HEADERS} "${_ARGS_CXX_OUT_DIR}/${_fil_we}.pb.h")
add_custom_command(
OUTPUT
"${_ARGS_CXX_OUT_DIR}/${_fil_we}.pb.cc"
"${_ARGS_CXX_OUT_DIR}/${_fil_we}.pb.h"
COMMAND
${_ARGS_PROTOC}
ARGS
--cpp_out=${_ARGS_CXX_OUT_DIR} ${_proto_include_cli_args} ${_abs_fil}
DEPENDS ${_abs_fil}
COMMENT "Running C++ protocol buffer compiler on ${_fil}"
VERBATIM
)
endforeach()
set_source_files_properties(
${${_ARGS_CXX_SOURCES}}
${${_ARGS_CXX_HEADERS}}
PROPERTIES
GENERATED TRUE
)
# remove some warnings from protobuf generated files
if(WIN32)
set_source_files_properties(
${${_ARGS_CXX_SOURCES}}
PROPERTIES
COMPILE_FLAGS "/wd4309 /wd4838"
)
else()
set_source_files_properties(
${${_ARGS_CXX_SOURCES}}
PROPERTIES
COMPILE_FLAGS "-Wno-unused-parameter -Wno-effc++ -Wno-pedantic -Wno-sign-compare -Wno-uninitialized"
)
endif()
set(${_ARGS_CXX_SOURCES} ${${_ARGS_CXX_SOURCES}} PARENT_SCOPE)
set(${_ARGS_CXX_HEADERS} ${${_ARGS_CXX_HEADERS}} PARENT_SCOPE)
endfunction()
+49
View File
@@ -0,0 +1,49 @@
include_guard(DIRECTORY)
function(protobuf_generate_fs)
set(_options)
set(_one_value_args TARGET_NAME OUT_DIR PROTOC)
set(_multi_value_args PROTO_INCLUDE_DIRS PROTO_SOURCES)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
if (NOT DEFINED _ARGS_TARGET_NAME)
set(_ARGS_TARGET_NAME "${PROJECT_NAME}_generate_fs_protos")
endif()
if (NOT DEFINED _ARGS_PROTOC)
set(_ARGS_PROTOC "${EXTERNALS_BIN}/protoc")
endif()
set(_proto_include_cli_args)
foreach(_dir ${_ARGS_PROTO_INCLUDE_DIRS})
list(APPEND _proto_include_cli_args -I ${_dir})
endforeach()
set(_outputs)
foreach(_proto_source ${_ARGS_PROTO_SOURCES})
# get_filename_component(_abs_proto_source ${_proto_source} ABSOLUTE)
get_filename_component(_proto_source_we ${_proto_source} NAME_WE)
# get_filename_component(_proto_source_dir ${_proto_source} DIRECTORY)
list(APPEND _outputs ${_ARGS_OUT_DIR}/${_proto_source_we}.fable.fs)
endforeach()
set(_venv_exec ${CMAKE_SOURCE_DIR}/tools/venv_exec.sh)
set(_plugin_py ${CMAKE_SOURCE_DIR}/src/lib_spark/spark/api/service_compiler/plugin.py)
set(_template_fs ${CMAKE_SOURCE_DIR}/src/lib_spark/spark/api/service_compiler/templates/fs/fable.fs)
add_custom_command(
OUTPUT ${_outputs}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMAND ${_venv_exec}
${_ARGS_PROTOC}
--plugin=protoc-gen-service=${_plugin_py}
--service_out=lang=fs,strip_out_path=1:${_ARGS_OUT_DIR}
${_proto_include_cli_args}
${_ARGS_PROTO_SOURCES}
DEPENDS
${_venv_exec}
${_plugin_py}
${_template_fs}
${CMAKE_CURRENT_SOURCE_DIR}/package-lock.json
${_ARGS_PROTO_SOURCES}
)
add_custom_target(${_ARGS_TARGET_NAME} ALL DEPENDS "${_outputs}")
endfunction()
+33
View File
@@ -0,0 +1,33 @@
include_guard(DIRECTORY)
function(protobuf_generate_js)
set(_options)
set(_one_value_args TARGET_NAME OUTPUT)
set(_multi_value_args PROTO_INCLUDE_DIRS PROTO_SOURCES)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
if (NOT DEFINED _ARGS_TARGET_NAME)
set(_ARGS_TARGET_NAME "${PROJECT_NAME}_generate_js_protos")
endif()
set(_proto_include_cli_args)
foreach(_dir ${_ARGS_PROTO_INCLUDE_DIRS})
list(APPEND _proto_include_cli_args -p ${_dir})
endforeach()
add_custom_command(
OUTPUT ${_ARGS_OUTPUT}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
COMMAND npx
--no-install pbjs
-t json-module
-w commonjs
-o ${_ARGS_OUTPUT}
${_proto_include_cli_args}
${_ARGS_PROTO_SOURCES}
DEPENDS
${CMAKE_CURRENT_SOURCE_DIR}/package-lock.json
${_ARGS_PROTO_SOURCES}
)
add_custom_target(${_ARGS_TARGET_NAME} ALL DEPENDS ${_ARGS_OUTPUT})
endfunction()
+51
View File
@@ -0,0 +1,51 @@
include_guard(DIRECTORY)
include(${CMAKE_SOURCE_DIR}/cmake/TernaryOperator.cmake)
function(sym_link_dir)
set(_options)
set(_one_value_args TARGET_NAME SOURCE DEST COMMENT)
set(_multi_value_args)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
add_custom_command(
OUTPUT ${_ARGS_DEST}
COMMAND ln ARGS -sf ${_ARGS_SOURCE} ${_ARGS_DEST}
)
set_ternary(_comment _ARGS_COMMENT "${_ARGS_COMMENT}" "linking dir ${_ARGS_SOURCE} -> ${_ARGS_DEST}")
add_custom_target(
${_ARGS_TARGET_NAME} ALL
DEPENDS ${_ARGS_DEST}
COMMENT "${_comment}"
)
endfunction()
function(sym_link_file)
set(_options)
set(_one_value_args TARGET_NAME SOURCE DEST INSTALL_DEST)
set(_multi_value_args)
cmake_parse_arguments(_ARGS "${_options}" "${_one_value_args}" "${_multi_value_args}" ${ARGN})
add_custom_command(
OUTPUT ${_ARGS_DEST}
COMMAND ln ARGS -s ${_ARGS_SOURCE} ${_ARGS_DEST}
)
add_custom_target(
${_ARGS_TARGET_NAME} ALL
DEPENDS ${_ARGS_DEST}
)
if (DEFINED _ARGS_INSTALL_DEST)
install(
FILES
${_ARGS_SOURCE}
DESTINATION ${_ARGS_INSTALL_DEST}
)
endif()
endfunction()
function(sym_link_local_lib LIB_NAME)
set(_lib_file_name "${CMAKE_SHARED_LIBRARY_PREFIX}${LIB_NAME}.so")
file(
CREATE_LINK
"${CMAKE_BINARY_DIR}/src/lib_${LIB_NAME}/${_lib_file_name}"
"${CMAKE_CURRENT_BINARY_DIR}/${_lib_file_name}"
SYMBOLIC
)
endfunction()
+9
View File
@@ -0,0 +1,9 @@
include_guard(GLOBAL)
macro(set_ternary RESULT CONDITION TRUE_VALUE FALSE_VALUE)
if(${CONDITION})
set(${RESULT} ${TRUE_VALUE})
else()
set(${RESULT} ${FALSE_VALUE})
endif()
endmacro()
+51
View File
@@ -0,0 +1,51 @@
services:
sparkd:
build:
dockerfile: Dockerfile
context: ./sparkd
command: [ "./sparkd", "run" ]
ports: [ "8000:8000", "8001:8001", "8002:8002" ]
pubsub:
build:
dockerfile: pubsub.Dockerfile
context: ./DataStream
command: [ "./PubSubServer" ]
environment:
- DOTNET_ENVIRONMENT=Production
ports:
- 22019:22019
md-publisher:
build:
dockerfile: md-publisher.Dockerfile
context: ./DataStream
depends_on:
- pubsub
- sparkd
command:
[
"dockerize",
"-wait",
"tcp://pubsub:22019",
"-wait",
"tcp://sparkd:8000",
"-timeout",
"10s",
"./DataStreamPublisher"
]
environment:
- DOTNET_ENVIRONMENT=MarketData
webapp:
ports:
- 8080:8080
- 8443:8443
build:
dockerfile: webapp.Dockerfile
context: ./DataStream
command: [ "./Bts.DataStreamWebApp.Server" ]
environment:
- ASPNETCORE_HTTP_PORT=8080
- ASPNETCORE_HTTPS_PORT=8443
- ASPNETCORE_ENVIRONMENT=Staging
+72
View File
@@ -0,0 +1,72 @@
services:
sparkd:
build:
dockerfile: src/server/sparkd/Dockerfile
command: [ "./sparkd", "run" ]
ports: [ "8000:8000", "8001:8001", "8002:8002" ]
pubsub:
environment:
- DOTNET_ENVIRONMENT=Staging
build:
dockerfile: src/DataStreamSuite/PubSubServer/Dockerfile
ports:
- 22019:22019
command: [ "./PubSubServer" ]
md-publisher:
build:
dockerfile: src/DataStreamSuite/DataStreamPublisher/Dockerfile
depends_on:
- pubsub
- sparkd
command:
[
"dockerize",
"-wait",
"tcp://pubsub:22019",
"-wait",
"tcp://sparkd:8000",
"-timeout",
"10s",
"./DataStreamPublisher"
]
environment:
- DOTNET_ENVIRONMENT=MarketData
webapp:
ports:
- 8080:8080
- 8443:8443
build:
dockerfile: src/DataStreamSuite/Bts.DataStreamWebApp.Server/Dockerfile
command: [ "./Bts.DataStreamWebApp.Server" ]
environment:
- ASPNETCORE_ENVIRONMENT=Staging
api-server:
profiles:
- api-server
depends_on:
- sparkd
ports:
- 8008:8008
- 9443:9443
build:
dockerfile: src/client/spark_api_server/Dockerfile
command: [ "./spark_api_server", "run" ]
environment:
- ASPNETCORE_ENVIRONMENT=Staging
sparkd-ping-test:
depends_on:
- sparkd
profiles:
- ping-test
build:
context: .
dockerfile_inline: |
FROM bluetradingsystems/linux_dotnet7
USER "root"
RUN apt-get update && apt-get -y install iputils-ping
command: [ "ping", "sparkd" ]
+1 -1
View File
@@ -53,7 +53,7 @@ The status callback is called when the market for this security opens, closes, o
// Subscribe to security trading status
void IAlgoServices::subscribe_status(SecurityKey);
// You implement
virtual void on_status(SecurityKey, co2::TradingStatus) override;
virtual void on_status(SecurityKey, domain::protobufs::TradingStatus) override;
```
# Child Orders
Submodule docker/linux deleted from 56c61231f3.
+40
View File
@@ -0,0 +1,40 @@
FROM ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y pkg-config \
&& apt-get install -y locales \
&& apt-get install -y ca-certificates \
&& apt-get install -y libssh2-1-dev \
&& apt-get install -y software-properties-common \
&& apt-get install -y sudo
# GCC
ARG gcc_major=13
RUN add-apt-repository ppa:ubuntu-toolchain-r/test \
&& apt-get update \
&& apt-get install -y g++-$gcc_major
RUN apt-get update \
&& apt-get install -y dotnet7
RUN locale-gen en_US.UTF-8
ARG docker_user="bts"
ARG docker_uid="1000"
ARG docker_pass="bts"
# ARG entrypoint
RUN \
groupadd -g "$docker_uid" "$docker_user" \
&& useradd \
-m \
-u "$docker_uid" \
-g "$docker_uid" \
-G sudo \
--password "$(openssl passwd -1 "$docker_pass")" \
"$docker_user"
USER bts
WORKDIR /home/bts
Submodule docker/windows deleted from de7d40c09d.
@@ -8,7 +8,6 @@
public int Integer;
public double Real;
public string Identifier;
public DateTime DateTime;
public ImmutableList<IArithmeticExpression> Arguments;
public ImmutableList<Assignment> Assignments;
public IArithmeticExpression Argument;
@@ -1,4 +1,6 @@
using System;
using System.Runtime.Versioning;
using Microsoft.Win32;
namespace BCG
@@ -10,6 +12,7 @@ namespace BCG
rk.SetValue (name, value.ToString ());
}
[SupportedOSPlatform("windows")]
public static double getValue (RegistryKey rk, string name)
{
object v = rk.GetValue (name);
@@ -1,19 +1,11 @@
namespace Bts.BooleanExpressions;
using bts;
class BooleanExpression : IBooleanExpression {
readonly Func<IDictionary<string, object?>, bool> evaluate;
public BooleanExpression(Func<IDictionary<string, object?>, bool> evaluate) =>
this.evaluate = evaluate;
public bool Evaluate(IDictionary<string, object?> env) {
try {
return evaluate(env);
} catch (Exception ex) {
Log.assert(false, $"BooleanExpression.Evaluate failed: {ex}");
throw;
}
}
public bool Evaluate(IDictionary<string, object?> env) =>
evaluate(env);
}
@@ -47,11 +47,6 @@ IsOfType '/'
%%
{StringConstant} {
yylval.String = yytext.Substring(1, yytext.Length - 2).Replace(@"\'", "'");
return (int)Token.STRING_CONSTANT;
}
{BooleanConstant} {
yylval.Boolean = yytext.ToLower() switch {
"true" => true,
@@ -62,6 +57,11 @@ IsOfType '/'
return (int)Token.BOOLEAN_CONSTANT;
}
{StringConstant} {
yylval.String = yytext.Substring(1, yytext.Length - 2).Replace(@"\'", "'");
return (int)Token.STRING_CONSTANT;
}
{NullConstant} {
yylval.Null = DBNull.Value;
return (int)Token.NULL_CONSTANT;
@@ -0,0 +1 @@
VITE_DATASTREAM_WEBSOCKET_URL=wss://localhost:443
@@ -0,0 +1 @@
VITE_DATASTREAM_WEBSOCKET_URL=wss://localhost:443
@@ -0,0 +1,32 @@
module.exports = {
root: true,
env: { browser: true, es2020: true },
extends: [
'eslint:recommended',
'plugin:react/recommended',
'plugin:react/jsx-runtime',
'plugin:react-hooks/recommended',
],
ignorePatterns: ['dist', '.eslintrc.cjs'],
parserOptions: { ecmaVersion: 'latest', sourceType: 'module' },
settings: { react: { version: '18.3.1' } },
plugins: ['react-refresh'],
rules: {
"import/order": [
"error",
{
"groups": [
"builtin",
"external",
"internal",
["parent", "sibling", "index"]
],
"newlines-between": "always"
}
],
'react-refresh/only-export-components': [
'warn',
{ allowConstantExport: true },
],
},
}
@@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
@@ -0,0 +1,11 @@
<Project Sdk="Microsoft.VisualStudio.JavaScript.Sdk/0.5.128-alpha">
<PropertyGroup>
<StartupCommand>npm run dev</StartupCommand>
<JavaScriptTestRoot>src\</JavaScriptTestRoot>
<JavaScriptTestFramework>Jest</JavaScriptTestFramework>
<!-- Allows the build (or compile) script located on package.json to run on Build -->
<ShouldRunBuildScript>true</ShouldRunBuildScript>
<!-- Folder where production build objects will be placed -->
<PublishAssetsDirectory>$(MSBuildProjectDirectory)\distill</PublishAssetsDirectory>
</PropertyGroup>
</Project>
@@ -0,0 +1,8 @@
# React + Vite
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
Currently, two official plugins are available:
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react/README.md) uses [Babel](https://babeljs.io/) for Fast Refresh
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh
@@ -0,0 +1,13 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/ico" href="/logo.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>BTS Cloud Streams</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.jsx"></script>
</body>
</html>
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
<disabledPackageSources>
<clear />
</disabledPackageSources>
</configuration>
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,44 @@
{
"name": "bts.datastreamwebapp.client",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"lint": "eslint . --ext js,jsx --report-unused-disable-directives --max-warnings 0",
"preview": "vite preview"
},
"dependencies": {
"@angular/material": "^18.0.4",
"@emotion/react": "^11.11.4",
"@emotion/styled": "^11.11.5",
"@fontsource/noto-sans": "^5.0.22",
"@material/data-table": "^15.0.0-canary.7f224ddd4.0",
"@mui/icons-material": "^5.15.20",
"@mui/material": "^5.15.20",
"@mui/x-data-grid": "^7.7.1",
"bulma": "^1.0.1",
"classnames": "^2.5.1",
"react": "^18.3.1",
"react-data-grid": "^7.0.0-beta.44",
"react-data-table-component": "^7.6.2",
"react-dom": "^18.3.1",
"react-json-view-lite": "^1.4.0",
"react-pro-sidebar": "^1.1.0",
"react-use-websocket": "^4.8.1",
"rpc-websockets": "^7.11.2"
},
"devDependencies": {
"@types/react": "^18.3.3",
"@types/react-dom": "^18.3.0",
"@vitejs/plugin-react": "^4.3.1",
"eslint": "^8.57.0",
"eslint-plugin-import": "^2.29.1",
"eslint-plugin-react": "^7.34.2",
"eslint-plugin-react-hooks": "^4.6.2",
"eslint-plugin-react-refresh": "^0.4.7",
"react-mui-sidebar": "^1.3.5",
"vite": "^5.2.13"
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>

After

Width:  |  Height:  |  Size: 1.5 KiB

@@ -0,0 +1,40 @@
import { Sidebar, Menu, MenuItem, Submenu, Logo } from "react-mui-sidebar";
import './App.css';
const AnimatedSideBar = () => {
return (
<Sidebar className={"sidebar-container"} width={"270px"}>
<Logo img="https://adminmart.com/wp-content/uploads/2024/03/logo-admin-mart-news.png">
AdminMart
</Logo>
<Menu subHeading="HOME">
<MenuItem link="/" badge="true">
Modern
</MenuItem>
<MenuItem>eCommerce</MenuItem>
<MenuItem>Analytical</MenuItem>
</Menu>
<Menu subHeading="APPS">
<MenuItem>Chat</MenuItem>
<MenuItem>Calendar</MenuItem>
</Menu>
<Menu subHeading="OTHERS">
<Submenu title="Menu Level">
<MenuItem>Post</MenuItem>
<MenuItem>Details</MenuItem>
<Submenu title="Level 2">
<MenuItem>new</MenuItem>
<MenuItem>Hello</MenuItem>
</Submenu>
</Submenu>
<MenuItem>Chip</MenuItem>
<MenuItem target="_blank" link="google.com">
External Link
</MenuItem>
</Menu>
</Sidebar>
);
};
export default AnimatedSideBar;
@@ -0,0 +1,101 @@
@use "@material/checkbox";
@use "@material/icon-button";
@use "@material/data-table/data-table";
.root-div {
display: flex;
flex-direction: row;
align-items: flex-start;
}
.main-content-div {
display: flex;
flex-direction: row;
}
.title-div {
width: 100%;
}
.container {
display: flex;
}
.container-column {
display: flex;
flex-direction: column;
}
.container-row {
display: flex;
flex-direction: row;
align-items: stretch;
}
box.query-input-form {
/* p: 2, display: "flex", flexFlow: "column", alignItems: "stretch" */
display: flex;
flex-flow: column;
align-items: stretch;
}
/* li.ps-menuitem-root:hover {
background-color: #ff9922;
} */
.container-row-centered {
display: flex;
flex-direction: row;
justify-content: center;
align-items: center;
}
h1#main-header {
padding: 4px 8px 4px 8px;
text-align: center;
}
td:hover {
background-color: cadetblue;
}
td#cell-decimal {
text-align: right;
}
td#cell-integer {
text-align: right;
}
td#cell-string {
text-align: center;
}
td#cell-date-time {
text-align: center;
}
/* Sidebar.css */
/* .sidebar-container {
display: flex;
flex-direction: column;
}
.sidebar {
position: relative;
top: 0;
left: 0;
height: 100vh;
background-color: #333;
color: white;
overflow: hidden;
transition: transform 0.3s ease;
}
.sidebar.closed {
width: 0;
}
.sidebar.open {
width: 250px;
} */
@@ -0,0 +1,173 @@
import { ThemeProvider, CssBaseline, GlobalStyles } from '@mui/material';
import QueryStatusView from './QueryStatusView';
import useMediaQuery from '@mui/material/useMediaQuery';
import QueryControl from './QueryControl';
import DataTable from './DataTable';
import { useState, useEffect, useMemo } from 'react';
import useWebSocket, { ReadyState } from 'react-use-websocket';
import Sidebar from './SideBar';
import getTheme from './Theme';
import '@fontsource/noto-sans';
import './App.css';
import './SideBar.css';
const webSocketUrl = import.meta.env.VITE_DATASTREAM_WEBSOCKET_URL;
// Always use the same subscription id (1), because we only show 1 table at a time.
// The server will automatically dispose the previous subscription and start a new one
// if we're currently subscribed to another query.
const subscriptionId = "1";
var rpcReqIdCounter = 1;
function subscribeReq(query) {
return {
jsonrpc: "2.0",
id: rpcReqIdCounter++,
method: "subscribe",
params: {
subscription_id: subscriptionId,
query: query
}
};
}
function unsubscribeReq() {
return {
jsonrpc: "2.0",
id: rpcReqIdCounter++,
method: "unsubscribe",
params: {
subscription_id: subscriptionId,
}
};
}
const App = () => {
const prefersDarkMode = useMediaQuery('(prefers-color-scheme: dark)');
const theme = getTheme({ mode: prefersDarkMode ? 'dark' : 'light' });
const globalStyles = {
':root': {
'--primary-color': theme.palette.primary.main,
'--primary-dark-color': theme.palette.primary.dark,
'--secondary-color': theme.palette.secondary.main,
'--secondary-dark-color': theme.palette.secondary.dark,
'--background-color': theme.palette.background.default,
},
};
const [blueTable, setBlueTable] = useState({ columns: null, rows: null });
const [queryError, setQueryError] = useState(null);
const [queryState, setQueryState] = useState("Unsubscribed");
const queryStateDetails = useMemo(() => {
if (queryState == "Unsubscribed") {
return "Enter query above, then click the play button to subscribe";
}
if (queryState == "Subscribed") {
return null;
}
}, [queryState]);
const [sidebarIsOpen, setSidebarIsOpen] = useState(false);
const [cannedQuery, setCannedQuery] = useState(null);
const onCannedQueryClicked = (cannedQuery) => {
console.info(`Executing canned query: ${JSON.stringify(cannedQuery)}`);
const queryText = cannedQuery.query.join("\n");
setCannedQuery({ queryText: queryText, clickTime: Date.now() });
subscribe(queryText);
};
const { sendJsonMessage, getWebSocket, readyState } = useWebSocket(
webSocketUrl,
{
onOpen: (e) => console.info(`WebSocket open. ${JSON.stringify(e)}`),
onClose: (e) => console.info(`WebSocket closed. ${JSON.stringify(e)}`),
onError: (e) => console.error(`WebSocket error: ${JSON.stringify(e)}`),
shouldReconnect: () => true,
});
useEffect(() => {
console.info(`WebSocket readyState: ${readyState}`)
if (readyState != ReadyState.OPEN) {
return;
}
const ws = getWebSocket();
ws.onmessage = e => {
console.info(`Received message: ${e.data}`);
const msg = JSON.parse(e.data);
setQueryError(msg.error);
if (msg.error != null) {
setQueryState("Error");
} else if (msg.id != null) {
setQueryState(msg.result.status);
} else {
console.assert(msg.params.subscription_id == subscriptionId);
console.info(`Received rows: ${msg.params.table.all}`);
setBlueTable({ columns: msg.params.table.columns, rows: msg.params.table.all });
}
};
}, [getWebSocket, readyState]);
const subscribe = (query) => {
const req = subscribeReq(query);
console.info(`Subscribing: ${JSON.stringify(subscribeReq(query))}`);
sendJsonMessage(req);
};
const unsubscribe = () => {
const req = unsubscribeReq();
console.info(`Unsubscribing: ${JSON.stringify(req)}`);
sendJsonMessage(req);
};
const onOpened = () => {
setSidebarIsOpen(true);
};
const onClosed = () => {
setSidebarIsOpen(false);
};
const sidebarWidth = sidebarIsOpen ? '250px' : '65px';
return (
<ThemeProvider theme={theme}>
<CssBaseline />
<GlobalStyles styles={globalStyles} />
<div className='root-div'>
<div style={{
display: 'flex',
flexDirection: 'column',
alignItems: 'flex-start',
flexWrap: 'nowrap',
width: sidebarWidth,
transition: 'width 0.3s ease',
}}>
<Sidebar onSelection={onCannedQueryClicked} onOpened={onOpened} onClosed={onClosed} />
</div>
<div style={{
display: 'flex',
flexDirection: 'column',
left: sidebarWidth,
width: '100%',
transition: 'width 0.3s ease',
}}>
<div style={{ width: '100%', display: 'flex', flexDirection: 'column', alignItems: 'flex-start', margin: theme.spacing(2, 0, 0, 0) }}>
<div style={{ width: '90%', rightMargin: '100px' }}>
<QueryControl
queryState={queryState}
queryText={cannedQuery?.queryText}
queryTextResetTime={cannedQuery?.clickTime}
subscribe={subscribe}
unsubscribe={unsubscribe} />
</div>
<div style={{ width: '90%', rightMargin: '100px' }}>
<QueryStatusView queryState={queryState} details={queryStateDetails} error={queryError} />
<DataTable columns={blueTable?.columns ?? []} rows={blueTable?.rows ?? []} />
</div>
</div>
</div>
</div>
</ThemeProvider>
);
};
export default App;
@@ -0,0 +1,53 @@
@use "@material/checkbox";
@use "@material/icon-button";
@use "@material/data-table/data-table";
.data-grid-container {
overflow: auto;
white-space: nowrap;
}
.data-grid {
max-height: 1000px;
}
div[role=columnheader] {
border-top: 0px;
border-bottom: 0px;
background-color: var(--primary-dark-color);
}
/* div[role=columnheader] {
border-top: 0px;
border-bottom: 0px;
background-color: var(--primary-dark-color);
}
.rdg-row {
border: 0px;
border-top: 0px;
border-bottom: 0px;
background-color: rgba(var(--primary-dark-color), 0.8);
color: var(--primary-color);
}
.rdg-row-odd {
border: 0px;
border-top: 0px;
border-bottom: 0px;
background-color: var(--primary-dark-color);
color: var(--primary-color);
}
.rdg-cell {
border-bottom: 0px;
}
.rdg-cell.even {
border-bottom: 5px;
}
.rdg-cell.odd {
border-bottom: 5px;
} */
@@ -0,0 +1,32 @@
@use "@material/checkbox";
@use "@material/icon-button";
@use "@material/data-table/data-table";
th {
font-size: large;
padding: 15px;
}
td {
padding: 15px;
}
td:hover {
background-color: cadetblue;
}
td#cell-decimal {
text-align: right;
}
td#cell-integer {
text-align: right;
}
td#cell-string {
text-align: center;
}
td#cell-date-time {
text-align: center;
}
@@ -0,0 +1,74 @@
import PropTypes from 'prop-types';
import { useTheme } from '@mui/material/styles';
import { DataGrid } from '@mui/x-data-grid';
import './DataGrid.css'
import { useMemo, useState } from 'react';
export default function DataTable({ columns, rows }) {
const theme = useTheme();
const [columnWidths, setColumnWidths] = useState({});
const handleColumnResize = (params) => {
const newWidth = params.width;
setColumnWidths((prevWidths) => ({
...prevWidths,
[params.colDef.field]: params.width,
}));
};
const getGridColDef = useMemo(() => {
return col => {
return {
field: col.name,
headerName: col.name,
width: columnWidths[col.name],
};
}
}, [columnWidths]);
return (
<DataGrid
className='data-grid-container'
autosizeOnMount={false}
columns={columns?.map(getGridColDef) ?? []}
onColumnResize={handleColumnResize}
rows={rows ?? []}
sx={{
margin: theme.spacing(2, 0, 0, 0),
borderColor: 'primary.main',
backgroundColor: 'primary.dark',
color: 'text.primary',
'& .MuiDataGrid-columnHeader': {
color: 'primary.main',
fontWeight: 700,
backgroundColor: 'primary.dark'
},
'& .MuiDataGrid-filler': {
color: 'primary.main',
fontWeight: 700,
backgroundColor: 'primary.dark'
},
'& .MuiDataGrid-cell': {
color: 'text.primary',
},
'& .MuiDataGrid-cell:hover': {
color: 'primary.main',
},
'& .MuiDataGrid-row': {
'--DataGrid-rowBorderColor': 'secondary.dark'
}
}}
/>
);
}
DataTable.propTypes = {
columns: PropTypes.arrayOf(
PropTypes.shape({
name: PropTypes.string.isRequired,
type: PropTypes.string.isRequired,
})
).isRequired,
rows: PropTypes.arrayOf(PropTypes.array).isRequired,
apiRef: PropTypes.object,
};
@@ -0,0 +1,119 @@
import PropTypes from 'prop-types';
import DataTable from './DataTable';
import { useRef, useState, useEffect } from 'react';
import { useGridApiRef } from '@mui/x-data-grid';
import Box from '@mui/material/Box';
import Button from '@mui/material/Button';
import Grid from '@mui/material/Grid';
import Paper from '@mui/material/Paper';
import PlayArrowIcon from '@mui/icons-material/PlayArrow';
import StopIcon from '@mui/icons-material/Stop';
import TextField from '@mui/material/TextField';
import Tooltip from '@mui/material/Tooltip';
const QueryControl = (props) => {
const { queryState, queryText, queryTextResetTime, subscribe, unsubscribe } = props;
// const theme = useTheme();
let textInput = useRef();
const [queryTextChangeTime, setQueryTextChangeTime] = useState(null);
useEffect(() => {
if (queryText == null || textInput.current == null) {
return;
}
// Init text field (unless a non-empty string is already present, or the user has modified
// the query - we don't want to overwrite their changes).
if (queryTextChangeTime == null || queryTextResetTime > queryTextChangeTime) {
console.info(`Initializing query text: ${queryText}`)
textInput.current.value = queryText;
}
}, [queryText, queryTextChangeTime, queryTextResetTime]);
const onClearClick = () => {
textInput.current.value = '';
};
const onRunClick = () => {
subscribe(textInput.current.value);
};
const onStopClick = () => {
unsubscribe();
};
const onKeyDown = (e) => {
// Detect keyboard shortcuts.
// Ctrl+Enter : Run
if (e.ctrlKey && e.key === 'Enter') {
onRunClick();
}
};
const onChange = (e) => {
// So we know not to overwrite the changes with the canned query on re-render
setQueryTextChangeTime(Date.now());
};
return (
<Box sx={{ display: "flex", flexFlow: "column" }}>
<Paper elevation={3} sx={{ color: 'primary.main' }}>
<Box sx={{ p: 2 }}>
<TextField
fullWidth
multiline
rows={10}
variant="outlined"
onChange={onChange}
onKeyDown={onKeyDown}
inputRef={textInput}
placeholder="Enter Query..."
/>
</Box>
<Box sx={{ p: 2, borderTop: 1 }}>
<Grid container spacing={2} justifyContent="flex-start">
<Grid item xs>
<Tooltip title="Clear Query Text">
<Button onClick={onClearClick} sx={{ padding: '11px', color: 'text.dark' }}>
Clear
</Button>
</Tooltip>
</Grid>
<Grid item>
<Tooltip title="Run (Ctrl+Enter)">
<Button onClick={onRunClick} sx={{
color: `queryStartButton.${queryState}.color`,
bgcolor: `queryStartButton.${queryState}.backgroundColor`,
padding: '11px'
}}>
<PlayArrowIcon />
</Button>
</Tooltip>
</Grid>
<Grid item>
<Tooltip title="Stop">
<Button onClick={onStopClick} sx={{
color: `queryStopButton.${queryState}.color`,
bgcolor: `queryStopButton.${queryState}.backgroundColor`,
padding: '11px'
}}>
<StopIcon />
</Button>
</Tooltip>
</Grid>
</Grid>
</Box>
</Paper>
</Box>
);
};
QueryControl.propTypes = {
columns: PropTypes.array,
rows: PropTypes.array,
isRunning: PropTypes.bool,
queryText: PropTypes.string,
queryTextResetTime: PropTypes.number,
queryState: PropTypes.string,
subscribe: PropTypes.func,
unsubscribe: PropTypes.func,
};
export default QueryControl;
@@ -0,0 +1,73 @@
import { useState } from 'react';
import PropTypes from 'prop-types';
import { Box, Typography, Collapse } from '@mui/material';
import Alert from '@mui/material/Alert';
import AlertTitle from '@mui/material/AlertTitle';
import { useTheme } from '@mui/material/styles';
import { alpha } from '@mui/system';
const QueryStatusView = ({ queryState, details, error }) => {
const theme = useTheme();
const [open, setOpen] = useState(false);
let header = queryState;
let headerContent = null;
let detailElements = [];
let severity = queryState == "Subscribed" ? "success" : "info";
let cursor = details == null ? "default" : "pointer";
if (error != null) {
severity = "error";
cursor = "pointer";
header = `Error Code: ${error.code}`;
headerContent = error.message;
detailElements.push(<Typography variant="body1">{error.data.error_message}</Typography>);
detailElements.push(<Typography variant="subtitle1">Stack Trace:</Typography>);
(error.data.stack_trace ?? []).forEach(line => detailElements.push(
<Typography variant="body2" component="pre" sx={{ whiteSpace: 'pre-wrap' }}>
{line}
</Typography>));
} else if (details != null) {
detailElements.push(<Typography variant="body1">{details}</Typography>);
}
return (
<Box sx={{ margin: theme.spacing(2, 0, 0, 0) }}>
<Alert
severity={severity}
onClick={() => setOpen(!open)}
sx={{
margin: theme.spacing(2, 0, 0, 0),
cursor: cursor,
}}>
<AlertTitle>{header}</AlertTitle>
{headerContent}
</Alert>
{detailElements.length == 0
? null
: <Collapse in={open}>
<Box
sx={{
backgroundColor: alpha(theme.palette[severity].light, 0.75),
color: theme.palette[severity].contrastText,
padding: theme.spacing(2),
marginTop: theme.spacing(0),
borderRadius: theme.shape.borderRadius,
}}
>
{detailElements}
</Box>
</Collapse>}
</Box>
);
};
QueryStatusView.propTypes = {
queryState: PropTypes.string,
details: PropTypes.string,
error: PropTypes.shape({
code: PropTypes.number,
message: PropTypes.string,
data: PropTypes.object,
})
};
export default QueryStatusView;
@@ -0,0 +1,27 @@
import 'react-data-grid/lib/styles.css';
import PropTypes from 'prop-types';
import DataGrid from 'react-data-grid';
import './DataGrid.css';
function getGridColDef(col) {
return {
key: col.name,
name: col.name,
};
}
export default function ReactDataGrid({columns, rows}) {
return (
<DataGrid className={"data-grid"} rows={rows} columns={columns.map(getGridColDef)} />
)
}
ReactDataGrid.propTypes = {
columns: PropTypes.arrayOf(
PropTypes.shape({
name: PropTypes.string.isRequired,
type: PropTypes.string.isRequired,
})
).isRequired,
rows: PropTypes.arrayOf(PropTypes.array).isRequired,
};
@@ -0,0 +1,98 @@
/* Sidebar.css */
.app-container {
display: flex;
width: 100%;
overflow: hidden;
flex-direction: row;
justify-content: flex-start;
align-items: flex-start;
}
.sidebar {
display: flex;
flex-direction: row;
justify-content: flex-start;
align-items: flex-start;
background-color: transparent;
color: white;
overflow: hidden;
transition: width 0.3s ease;
}
.sidebar.open {
width: 100%;
}
.sidebar.closed {
width: 0%;
}
.sidebar-toggle-button {
border: 0px;
border-radius: 50%;
box-shadow: rgba(9, 30, 66, 0.08) 0px 0px 0px 1px, rgba(9, 30, 66, 0.08) 0px 2px 4px 1px;
color: var(--primary-color);
cursor: pointer;
height: 24px;
opacity: 1;
outline: 0px;
padding: 0px;
background-color: transparent;
transition: background-color 100ms linear 0s, color 100ms linear 0s, opacity 300ms cubic-bezier(0.2, 0, 0, 1) 0s, transform 300ms cubic-bezier(0.2, 0, 0, 1) 0s;
width: 20%;
}
.sidebar-toggle-button:hover {
transition: background-color 100ms linear 0s, color 100ms linear 0s, opacity 300ms cubic-bezier(0.2, 0, 0, 1) 0s, transform 300ms cubic-bezier(0.2, 0, 0, 1) 0s;
}
.main-content.open {
width: 165px;
transition: width 0.3s ease;
background-color: transparent;
}
.main-content.closed {
width: 0%;
/* min-width: 25px; */
transition: width 0.3s ease;
background-color: transparent;
}
.sidebar-menu-item {
width: 100%;
box-sizing: border-box;
}
.sidebar-menu-item.open {
width: 100%;
/* justify-content: flex-start; */
transition: width 0.3s ease;
}
.sidebar-menu-item.closed {
width: 0%;
min-width: 0px;
/* justify-content: flex-start; */
transition: width 0.3s ease;
}
.sidebar-menu-item-content.open {
width: 100%;
/* justify-content: flex-start; */
transition: width 0.3s ease;
}
.sidebar-menu-item-content.closed {
width: 0px;
/* justify-content: flex-start; */
transition: width 0.3s ease;
}
/* Ensure span can shrink */
.button-text {
display: block; /* Make span a block element */
width: 100%; /* Ensure it takes the full width of its parent */
overflow: hidden; /* Hide any overflow content */
white-space: nowrap; /* Prevent text wrapping */
}
@@ -0,0 +1,109 @@
import { useState, useEffect } from 'react';
import Button from '@mui/material/Button';
import { useTheme } from '@mui/material/styles';
import PropTypes from 'prop-types';
import Tooltip from '@mui/material/Tooltip';
import ChevronLeftRounded from "@mui/icons-material/ChevronLeftRounded"
import ChevronRightRounded from '@mui/icons-material/ChevronRightRounded';
import FileCopyIcon from '@mui/icons-material/FileCopy';
import './SideBar.css';
const Sidebar = ({ onSelection, onOpened, onClosed }) => {
const theme = useTheme();
const [isOpen, setIsOpen] = useState(false);
const [cannedQueries, setCannedQueries] = useState([]);
const Icon = isOpen ? ChevronLeftRounded : ChevronRightRounded;
const toggleIsOpen = () => {
if (isOpen) {
setIsOpen(false);
if (onClosed != null) {
onClosed();
}
} else {
setIsOpen(true);
if (onOpened != null) {
onOpened();
}
}
};
const onMenuItemClick = (event, cannedQuery) => {
console.info(`menu item clicked: ${cannedQuery.name}`);
onSelection(cannedQuery);
};
// const onMenuItemClick = async (event, cannedQuery) => {
// try {
// await navigator.clipboard.writeText(cannedQuery.query.join("\n"));
// alert(`Copied ${cannedQuery.name} to clipboard`);
// } catch (err) {
// console.error('Failed to copy query text: ', err);
// }
// };
useEffect(() => {
const fetchCannedQueries = async () => {
try {
const response = await fetch('api/cannedqueries');
const data = await response.json();
setCannedQueries(data);
} catch (error) {
console.error('Error fetching WebSocket config:', error);
}
}
fetchCannedQueries();
}, []);
const mainContentClass = `main-content ${isOpen ? 'open' : 'closed'}`;
const menuItemClass = `sidebar-menu-item ${isOpen ? 'open' : 'closed'}`;
const menuItemContentClass = `sidebar-menu-item-content ${isOpen ? 'open' : 'closed'}`;
const toggleButtonTitle = `${isOpen ? 'Hide' : 'Show'} Common Queries`;
return (
<div
className={"sidebar"}
style={{
display: 'flex',
flexDirection: 'row',
justifyContent: 'flexStart',
alignItems: 'flexStart',
margin: theme.spacing(2, 0, 0, 0)
}}
>
<div className={mainContentClass}>
{cannedQueries.map(cq =>
<Button key={cq.name} className={menuItemClass} onClick={(event) => onMenuItemClick(event, cq)}>
<div className={menuItemContentClass}>
<span className='button-text'>{cq.name}</span>
</div>
</Button>)
}
{/* {cannedQueries.map(cq =>
<Button
variant="contained"
color="primary"
startIcon={<FileCopyIcon />}
onClick={onMenuItemClick}
>
<div className={menuItemContentClass}>
<span className='button-text'>{cq.name}</span>
</div>
</Button>)} */}
</div>
<Tooltip title={toggleButtonTitle}>
<Button onClick={toggleIsOpen}>
<Icon className="w-5 h-5" />
</Button>
</Tooltip>
</div>
);
};
export default Sidebar;
Sidebar.propTypes = {
onSelection: PropTypes.func,
onOpened: PropTypes.func,
onClosed: PropTypes.func,
};
@@ -0,0 +1,112 @@
import { createTheme } from '@mui/material/styles';
import PropTypes from 'prop-types';
import '@fontsource/noto-sans';
import './App.css';
const getLightPalette = () => {
return {
mode: 'light',
primary: {
light: '#6573c3',
main: '#3f51b5',
dark: '#2c387e',
contrastText: '#ffffff',
},
secondary: {
light: '#f73378',
main: '#f50057',
dark: '#ab003c',
contrastText: '#ffffff',
},
text: {
primary: '#000000c4',
secondary: '#00000086',
disabled: '#0000005b',
hint: '#0000005b',
}
};
};
const getDarkPalette = () => {
return {
mode: 'dark',
primary: {
light: '#bcc5ff',
main: '#33baf9',
dark: '#202334',
contrastText: '#78a2ac',
},
secondary: {
light: '#9fd0b8',
main: '#5eb289',
dark: '#ffffff12',
contrastText: '#c8d4f7',
},
action: {
hoverOpacity: 0.08
},
text: {
primary: '#eaf8f8',
dark: '#eaf8f8a0'
},
queryStartButton: {
Subscribed: {
color: '#33baf9',
backgroundColor: '#ffffff40',
},
Unsubscribed: {
color: '#ffffff40',
backgroundColor: '#ffffff00',
}
},
queryStopButton: {
Subscribed: {
color: '#ff4444a0',
backgroundColor: '#ffffff40',
},
Unsubscribed: {
color: '#ff444450',
backgroundColor: '#ffffff00',
}
},
error: {
light: '#e57373',
main: '#f44336',
dark: '#d32f2f',
contrastText: '#fff',
},
warning: {
light: '#ffb74d',
main: '#ff9800',
dark: '#f57c00',
contrastText: '#000',
},
info: {
light: '#64b5f6',
main: '#2196f3',
dark: '#303030',
contrastText: '#fff',
},
success: {
light: '#81c784',
main: '#4caf50',
dark: '#388e3c',
contrastText: '#000',
},
};
};
const getTheme = ({ mode }) => {
return createTheme({
typography: {
fontFamily: '"Helvetica"'
},
palette: mode == 'dark' ? getDarkPalette() : getLightPalette()
});
};
export default getTheme;
getTheme.propTypes = {
mode: PropTypes.string.isRequired,
};
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>

After

Width:  |  Height:  |  Size: 4.0 KiB

@@ -0,0 +1,51 @@
.dark-high-contrast {
--md-sys-color-primary: rgb(251 250 255);
--md-sys-color-surface-tint: rgb(165 200 255);
--md-sys-color-on-primary: rgb(0 0 0);
--md-sys-color-primary-container: rgb(173 204 255);
--md-sys-color-on-primary-container: rgb(0 0 0);
--md-sys-color-secondary: rgb(250 250 255);
--md-sys-color-on-secondary: rgb(0 0 0);
--md-sys-color-secondary-container: rgb(159 208 255);
--md-sys-color-on-secondary-container: rgb(0 0 0);
--md-sys-color-tertiary: rgb(255 249 251);
--md-sys-color-on-tertiary: rgb(0 0 0);
--md-sys-color-tertiary-container: rgb(222 193 231);
--md-sys-color-on-tertiary-container: rgb(0 0 0);
--md-sys-color-error: rgb(255 249 249);
--md-sys-color-on-error: rgb(0 0 0);
--md-sys-color-error-container: rgb(255 186 177);
--md-sys-color-on-error-container: rgb(0 0 0);
--md-sys-color-background: rgb(17 19 24);
--md-sys-color-on-background: rgb(225 226 233);
--md-sys-color-surface: rgb(17 19 24);
--md-sys-color-on-surface: rgb(255 255 255);
--md-sys-color-surface-variant: rgb(67 71 78);
--md-sys-color-on-surface-variant: rgb(251 250 255);
--md-sys-color-outline: rgb(200 202 212);
--md-sys-color-outline-variant: rgb(200 202 212);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(225 226 233);
--md-sys-color-inverse-on-surface: rgb(0 0 0);
--md-sys-color-inverse-primary: rgb(0 42 83);
--md-sys-color-primary-fixed: rgb(219 231 255);
--md-sys-color-on-primary-fixed: rgb(0 0 0);
--md-sys-color-primary-fixed-dim: rgb(173 204 255);
--md-sys-color-on-primary-fixed-variant: rgb(0 22 49);
--md-sys-color-secondary-fixed: rgb(214 233 255);
--md-sys-color-on-secondary-fixed: rgb(0 0 0);
--md-sys-color-secondary-fixed-dim: rgb(159 208 255);
--md-sys-color-on-secondary-fixed-variant: rgb(0 24 42);
--md-sys-color-tertiary-fixed: rgb(249 222 255);
--md-sys-color-on-tertiary-fixed: rgb(0 0 0);
--md-sys-color-tertiary-fixed-dim: rgb(222 193 231);
--md-sys-color-on-tertiary-fixed-variant: rgb(33 14 42);
--md-sys-color-surface-dim: rgb(17 19 24);
--md-sys-color-surface-bright: rgb(55 57 62);
--md-sys-color-surface-container-lowest: rgb(12 14 19);
--md-sys-color-surface-container-low: rgb(25 28 32);
--md-sys-color-surface-container: rgb(29 32 36);
--md-sys-color-surface-container-high: rgb(40 42 47);
--md-sys-color-surface-container-highest: rgb(50 53 58);
}
@@ -0,0 +1,51 @@
.dark-medium-contrast {
--md-sys-color-primary: rgb(173 204 255);
--md-sys-color-surface-tint: rgb(165 200 255);
--md-sys-color-on-primary: rgb(0 22 49);
--md-sys-color-primary-container: rgb(112 146 198);
--md-sys-color-on-primary-container: rgb(0 0 0);
--md-sys-color-secondary: rgb(159 208 255);
--md-sys-color-on-secondary: rgb(0 24 42);
--md-sys-color-secondary-container: rgb(100 149 193);
--md-sys-color-on-secondary-container: rgb(0 0 0);
--md-sys-color-tertiary: rgb(222 193 231);
--md-sys-color-on-tertiary: rgb(33 14 42);
--md-sys-color-tertiary-container: rgb(162 136 171);
--md-sys-color-on-tertiary-container: rgb(0 0 0);
--md-sys-color-error: rgb(255 186 177);
--md-sys-color-on-error: rgb(55 0 1);
--md-sys-color-error-container: rgb(255 84 73);
--md-sys-color-on-error-container: rgb(0 0 0);
--md-sys-color-background: rgb(17 19 24);
--md-sys-color-on-background: rgb(225 226 233);
--md-sys-color-surface: rgb(17 19 24);
--md-sys-color-on-surface: rgb(251 250 255);
--md-sys-color-surface-variant: rgb(67 71 78);
--md-sys-color-on-surface-variant: rgb(200 202 212);
--md-sys-color-outline: rgb(160 163 171);
--md-sys-color-outline-variant: rgb(128 131 139);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(225 226 233);
--md-sys-color-inverse-on-surface: rgb(40 42 47);
--md-sys-color-inverse-primary: rgb(35 73 120);
--md-sys-color-primary-fixed: rgb(212 227 255);
--md-sys-color-on-primary-fixed: rgb(0 17 40);
--md-sys-color-primary-fixed-dim: rgb(165 200 255);
--md-sys-color-on-primary-fixed-variant: rgb(10 55 100);
--md-sys-color-secondary-fixed: rgb(206 229 255);
--md-sys-color-on-secondary-fixed: rgb(0 18 34);
--md-sys-color-secondary-fixed-dim: rgb(154 203 250);
--md-sys-color-on-secondary-fixed-variant: rgb(0 57 91);
--md-sys-color-tertiary-fixed: rgb(247 216 255);
--md-sys-color-on-tertiary-fixed: rgb(28 9 37);
--md-sys-color-tertiary-fixed-dim: rgb(218 189 226);
--md-sys-color-on-tertiary-fixed-variant: rgb(67 46 76);
--md-sys-color-surface-dim: rgb(17 19 24);
--md-sys-color-surface-bright: rgb(55 57 62);
--md-sys-color-surface-container-lowest: rgb(12 14 19);
--md-sys-color-surface-container-low: rgb(25 28 32);
--md-sys-color-surface-container: rgb(29 32 36);
--md-sys-color-surface-container-high: rgb(40 42 47);
--md-sys-color-surface-container-highest: rgb(50 53 58);
}
@@ -0,0 +1,51 @@
.dark {
--md-sys-color-primary: rgb(165 200 255);
--md-sys-color-surface-tint: rgb(165 200 255);
--md-sys-color-on-primary: rgb(0 49 94);
--md-sys-color-primary-container: rgb(34 72 118);
--md-sys-color-on-primary-container: rgb(212 227 255);
--md-sys-color-secondary: rgb(154 203 250);
--md-sys-color-on-secondary: rgb(0 51 83);
--md-sys-color-secondary-container: rgb(12 74 114);
--md-sys-color-on-secondary-container: rgb(206 229 255);
--md-sys-color-tertiary: rgb(218 189 226);
--md-sys-color-on-tertiary: rgb(61 41 70);
--md-sys-color-tertiary-container: rgb(85 63 93);
--md-sys-color-on-tertiary-container: rgb(247 216 255);
--md-sys-color-error: rgb(255 180 171);
--md-sys-color-on-error: rgb(105 0 5);
--md-sys-color-error-container: rgb(147 0 10);
--md-sys-color-on-error-container: rgb(255 218 214);
--md-sys-color-background: rgb(17 19 24);
--md-sys-color-on-background: rgb(225 226 233);
--md-sys-color-surface: rgb(17 19 24);
--md-sys-color-on-surface: rgb(225 226 233);
--md-sys-color-surface-variant: rgb(67 71 78);
--md-sys-color-on-surface-variant: rgb(195 198 207);
--md-sys-color-outline: rgb(141 145 153);
--md-sys-color-outline-variant: rgb(67 71 78);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(225 226 233);
--md-sys-color-inverse-on-surface: rgb(46 48 53);
--md-sys-color-inverse-primary: rgb(60 96 144);
--md-sys-color-primary-fixed: rgb(212 227 255);
--md-sys-color-on-primary-fixed: rgb(0 28 58);
--md-sys-color-primary-fixed-dim: rgb(165 200 255);
--md-sys-color-on-primary-fixed-variant: rgb(34 72 118);
--md-sys-color-secondary-fixed: rgb(206 229 255);
--md-sys-color-on-secondary-fixed: rgb(0 29 50);
--md-sys-color-secondary-fixed-dim: rgb(154 203 250);
--md-sys-color-on-secondary-fixed-variant: rgb(12 74 114);
--md-sys-color-tertiary-fixed: rgb(247 216 255);
--md-sys-color-on-tertiary-fixed: rgb(39 20 48);
--md-sys-color-tertiary-fixed-dim: rgb(218 189 226);
--md-sys-color-on-tertiary-fixed-variant: rgb(85 63 93);
--md-sys-color-surface-dim: rgb(17 19 24);
--md-sys-color-surface-bright: rgb(55 57 62);
--md-sys-color-surface-container-lowest: rgb(12 14 19);
--md-sys-color-surface-container-low: rgb(25 28 32);
--md-sys-color-surface-container: rgb(29 32 36);
--md-sys-color-surface-container-high: rgb(40 42 47);
--md-sys-color-surface-container-highest: rgb(50 53 58);
}
@@ -0,0 +1,51 @@
.light-high-contrast {
--md-sys-color-primary: rgb(0 34 70);
--md-sys-color-surface-tint: rgb(60 96 144);
--md-sys-color-on-primary: rgb(255 255 255);
--md-sys-color-primary-container: rgb(29 68 114);
--md-sys-color-on-primary-container: rgb(255 255 255);
--md-sys-color-secondary: rgb(0 36 61);
--md-sys-color-on-secondary: rgb(255 255 255);
--md-sys-color-secondary-container: rgb(3 70 110);
--md-sys-color-on-secondary-container: rgb(255 255 255);
--md-sys-color-tertiary: rgb(46 26 55);
--md-sys-color-on-tertiary: rgb(255 255 255);
--md-sys-color-tertiary-container: rgb(81 59 89);
--md-sys-color-on-tertiary-container: rgb(255 255 255);
--md-sys-color-error: rgb(78 0 2);
--md-sys-color-on-error: rgb(255 255 255);
--md-sys-color-error-container: rgb(140 0 9);
--md-sys-color-on-error-container: rgb(255 255 255);
--md-sys-color-background: rgb(249 249 255);
--md-sys-color-on-background: rgb(25 28 32);
--md-sys-color-surface: rgb(249 249 255);
--md-sys-color-on-surface: rgb(0 0 0);
--md-sys-color-surface-variant: rgb(224 226 236);
--md-sys-color-on-surface-variant: rgb(32 36 43);
--md-sys-color-outline: rgb(63 67 74);
--md-sys-color-outline-variant: rgb(63 67 74);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(46 48 53);
--md-sys-color-inverse-on-surface: rgb(255 255 255);
--md-sys-color-inverse-primary: rgb(228 236 255);
--md-sys-color-primary-fixed: rgb(29 68 114);
--md-sys-color-on-primary-fixed: rgb(255 255 255);
--md-sys-color-primary-fixed-dim: rgb(0 45 88);
--md-sys-color-on-primary-fixed-variant: rgb(255 255 255);
--md-sys-color-secondary-fixed: rgb(3 70 110);
--md-sys-color-on-secondary-fixed: rgb(255 255 255);
--md-sys-color-secondary-fixed-dim: rgb(0 47 77);
--md-sys-color-on-secondary-fixed-variant: rgb(255 255 255);
--md-sys-color-tertiary-fixed: rgb(81 59 89);
--md-sys-color-on-tertiary-fixed: rgb(255 255 255);
--md-sys-color-tertiary-fixed-dim: rgb(57 37 66);
--md-sys-color-on-tertiary-fixed-variant: rgb(255 255 255);
--md-sys-color-surface-dim: rgb(217 218 224);
--md-sys-color-surface-bright: rgb(249 249 255);
--md-sys-color-surface-container-lowest: rgb(255 255 255);
--md-sys-color-surface-container-low: rgb(243 243 250);
--md-sys-color-surface-container: rgb(237 237 244);
--md-sys-color-surface-container-high: rgb(231 232 238);
--md-sys-color-surface-container-highest: rgb(225 226 233);
}
@@ -0,0 +1,51 @@
.light-medium-contrast {
--md-sys-color-primary: rgb(29 68 114);
--md-sys-color-surface-tint: rgb(60 96 144);
--md-sys-color-on-primary: rgb(255 255 255);
--md-sys-color-primary-container: rgb(83 118 167);
--md-sys-color-on-primary-container: rgb(255 255 255);
--md-sys-color-secondary: rgb(3 70 110);
--md-sys-color-on-secondary: rgb(255 255 255);
--md-sys-color-secondary-container: rgb(71 121 163);
--md-sys-color-on-secondary-container: rgb(255 255 255);
--md-sys-color-tertiary: rgb(81 59 89);
--md-sys-color-on-tertiary: rgb(255 255 255);
--md-sys-color-tertiary-container: rgb(133 108 141);
--md-sys-color-on-tertiary-container: rgb(255 255 255);
--md-sys-color-error: rgb(140 0 9);
--md-sys-color-on-error: rgb(255 255 255);
--md-sys-color-error-container: rgb(218 52 46);
--md-sys-color-on-error-container: rgb(255 255 255);
--md-sys-color-background: rgb(249 249 255);
--md-sys-color-on-background: rgb(25 28 32);
--md-sys-color-surface: rgb(249 249 255);
--md-sys-color-on-surface: rgb(25 28 32);
--md-sys-color-surface-variant: rgb(224 226 236);
--md-sys-color-on-surface-variant: rgb(63 67 74);
--md-sys-color-outline: rgb(91 95 103);
--md-sys-color-outline-variant: rgb(119 123 131);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(46 48 53);
--md-sys-color-inverse-on-surface: rgb(240 240 247);
--md-sys-color-inverse-primary: rgb(165 200 255);
--md-sys-color-primary-fixed: rgb(83 118 167);
--md-sys-color-on-primary-fixed: rgb(255 255 255);
--md-sys-color-primary-fixed-dim: rgb(58 93 141);
--md-sys-color-on-primary-fixed-variant: rgb(255 255 255);
--md-sys-color-secondary-fixed: rgb(71 121 163);
--md-sys-color-on-secondary-fixed: rgb(255 255 255);
--md-sys-color-secondary-fixed-dim: rgb(43 96 137);
--md-sys-color-on-secondary-fixed-variant: rgb(255 255 255);
--md-sys-color-tertiary-fixed: rgb(133 108 141);
--md-sys-color-on-tertiary-fixed: rgb(255 255 255);
--md-sys-color-tertiary-fixed-dim: rgb(107 84 116);
--md-sys-color-on-tertiary-fixed-variant: rgb(255 255 255);
--md-sys-color-surface-dim: rgb(217 218 224);
--md-sys-color-surface-bright: rgb(249 249 255);
--md-sys-color-surface-container-lowest: rgb(255 255 255);
--md-sys-color-surface-container-low: rgb(243 243 250);
--md-sys-color-surface-container: rgb(237 237 244);
--md-sys-color-surface-container-high: rgb(231 232 238);
--md-sys-color-surface-container-highest: rgb(225 226 233);
}
@@ -0,0 +1,51 @@
.light {
--md-sys-color-primary: rgb(60 96 144);
--md-sys-color-surface-tint: rgb(60 96 144);
--md-sys-color-on-primary: rgb(255 255 255);
--md-sys-color-primary-container: rgb(212 227 255);
--md-sys-color-on-primary-container: rgb(0 28 58);
--md-sys-color-secondary: rgb(46 98 140);
--md-sys-color-on-secondary: rgb(255 255 255);
--md-sys-color-secondary-container: rgb(206 229 255);
--md-sys-color-on-secondary-container: rgb(0 29 50);
--md-sys-color-tertiary: rgb(110 86 118);
--md-sys-color-on-tertiary: rgb(255 255 255);
--md-sys-color-tertiary-container: rgb(247 216 255);
--md-sys-color-on-tertiary-container: rgb(39 20 48);
--md-sys-color-error: rgb(186 26 26);
--md-sys-color-on-error: rgb(255 255 255);
--md-sys-color-error-container: rgb(255 218 214);
--md-sys-color-on-error-container: rgb(65 0 2);
--md-sys-color-background: rgb(249 249 255);
--md-sys-color-on-background: rgb(25 28 32);
--md-sys-color-surface: rgb(249 249 255);
--md-sys-color-on-surface: rgb(25 28 32);
--md-sys-color-surface-variant: rgb(224 226 236);
--md-sys-color-on-surface-variant: rgb(67 71 78);
--md-sys-color-outline: rgb(116 119 127);
--md-sys-color-outline-variant: rgb(195 198 207);
--md-sys-color-shadow: rgb(0 0 0);
--md-sys-color-scrim: rgb(0 0 0);
--md-sys-color-inverse-surface: rgb(46 48 53);
--md-sys-color-inverse-on-surface: rgb(240 240 247);
--md-sys-color-inverse-primary: rgb(165 200 255);
--md-sys-color-primary-fixed: rgb(212 227 255);
--md-sys-color-on-primary-fixed: rgb(0 28 58);
--md-sys-color-primary-fixed-dim: rgb(165 200 255);
--md-sys-color-on-primary-fixed-variant: rgb(34 72 118);
--md-sys-color-secondary-fixed: rgb(206 229 255);
--md-sys-color-on-secondary-fixed: rgb(0 29 50);
--md-sys-color-secondary-fixed-dim: rgb(154 203 250);
--md-sys-color-on-secondary-fixed-variant: rgb(12 74 114);
--md-sys-color-tertiary-fixed: rgb(247 216 255);
--md-sys-color-on-tertiary-fixed: rgb(39 20 48);
--md-sys-color-tertiary-fixed-dim: rgb(218 189 226);
--md-sys-color-on-tertiary-fixed-variant: rgb(85 63 93);
--md-sys-color-surface-dim: rgb(217 218 224);
--md-sys-color-surface-bright: rgb(249 249 255);
--md-sys-color-surface-container-lowest: rgb(255 255 255);
--md-sys-color-surface-container-low: rgb(243 243 250);
--md-sys-color-surface-container: rgb(237 237 244);
--md-sys-color-surface-container-high: rgb(231 232 238);
--md-sys-color-surface-container-highest: rgb(225 226 233);
}
@@ -0,0 +1,107 @@
:root {
font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif;
line-height: 1.5;
font-weight: 400;
color-scheme: light dark;
color: rgba(255, 255, 255, 0.87);
background-color: #242424;
font-synthesis: none;
text-rendering: optimizeLegibility;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
}
a {
font-weight: 500;
color: #646cff;
text-decoration: inherit;
}
a:hover {
color: #535bf2;
}
body {
margin: 0;
display: flex;
place-items: center;
min-width: 320px;
min-height: 100vh;
}
button {
border-radius: 8px;
border: 1px solid transparent;
padding: 0.6em 1.2em;
font-size: 1em;
font-weight: 500;
font-family: inherit;
background-color: #1a1a1a;
cursor: pointer;
transition: border-color 0.25s;
}
button:hover {
border-color: #646cff;
}
button:focus,
button:focus-visible {
outline: 4px auto -webkit-focus-ring-color;
}
button#query-button {
height: 5em;
vertical-align: top;
}
button#play-button {
padding: '10px 20px';
font-size: '16px';
cursor: 'pointer';
background-color: '#28a745';
color: '#fff';
border: 'none';
border-radius: '5px';
};
button#pause-button {
background-color: #ffc107;
}
button#stop-button {
background-color: #dc3545;
}
h1 {
font-size: 3.2em;
line-height: 1.1;
}
p#footer {
float: none;
clear: both;
vertical-align: bottom;
}
table#query-output {
float: none;
clear: both;
}
textarea#query-input {
height: 5em;
width: 50em;
}
@media (prefers-color-scheme: light) {
:root {
color: #213547;
background-color: #ffffff;
}
a:hover {
color: #747bff;
}
button {
background-color: #f9f9f9;
}
}
@@ -0,0 +1,7 @@
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
ReactDOM.createRoot(document.getElementById('root')).render(
<App />
)
@@ -0,0 +1,68 @@
import { fileURLToPath, URL } from 'node:url';
import { defineConfig, loadEnv } from 'vite';
import plugin from '@vitejs/plugin-react';
import fs from 'fs';
import path from 'path';
import child_process from 'child_process';
import process, { env } from 'process';
const baseFolder =
process.env.APPDATA !== undefined && process.env.APPDATA !== ''
? `${process.env.APPDATA}/ASP.NET/https`
: `${process.env.HOME}/.aspnet/https`;
const certificateName = "bts.datastreamwebapp.client";
const certFilePath = path.join(baseFolder, `${certificateName}.pem`);
const keyFilePath = path.join(baseFolder, `${certificateName}.key`);
if (!fs.existsSync(certFilePath) || !fs.existsSync(keyFilePath)) {
if (0 !== child_process.spawnSync('dotnet', [
'dev-certs',
'https',
'--export-path',
certFilePath,
'--format',
'Pem',
'--no-password',
], { stdio: 'inherit', }).status) {
throw new Error("Could not create certificate.");
}
}
export default defineConfig(({ $$command, mode, $$isSsrBuild, $$isPreview }) => {
const target = env.ASPNETCORE_HTTPS_PORT
? `https://localhost:${env.ASPNETCORE_HTTPS_PORT}`
: env.ASPNETCORE_URLS
? env.ASPNETCORE_URLS.split(';')[0]
: 'https://localhost:8443';
return {
plugins: [plugin()],
resolve: {
alias: {
'@': fileURLToPath(new URL('./src', import.meta.url))
}
},
build: {
minify: false,
sourcemap: true,
outDir: `../../../build/Bts.DataStreamWebApp.Client/${mode}/dist`
},
server: {
proxy: {
'^/api': {
target,
secure: false
},
'^/ws': {
target,
secure: false
}
},
port: 5173,
https: {
key: fs.readFileSync(keyFilePath),
cert: fs.readFileSync(certFilePath),
}
}
};
});
@@ -0,0 +1,40 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<InvariantGlobalization>true</InvariantGlobalization>
<SpaRoot>..\Bts.DataStreamWebApp.Client</SpaRoot>
<SpaProxyLaunchCommand>npm run dev</SpaProxyLaunchCommand>
<SpaProxyServerUrl>https://localhost:5173</SpaProxyServerUrl>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CommandLineParser" Version="2.9.1" />
<PackageReference Include="Microsoft.AspNetCore.SpaProxy" Version="7.0.20" />
<PackageReference Include="Microsoft.AspNetCore.SpaServices.Extensions" Version="7.0.20" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.4.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Bts.DataStreamWebApp.Client\Bts.DataStreamWebApp.Client.esproj">
<ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>
<ProjectReference Include="..\DataStream\DataStream.csproj" />
<ProjectReference Include="..\SharedStreams\SharedStreams.csproj" />
<ProjectReference Include="..\..\client\spark_api\spark_api.csproj" />
<ProjectReference Include="..\..\client\spark_core\spark_core.csproj" />
<ProjectReference Include="..\..\client\spark_websocket\spark_websocket.csproj" />
</ItemGroup>
<ItemGroup>
<ConfigurationProfile Include="appsettings.Development.json" />
<ConfigurationProfile Include="appsettings.Production.json" />
<ConfigurationProfile Include="appsettings.ProductionAws.json" />
<ConfigurationProfile Include="appsettings.Staging.json" />
<ConfigurationProfile Include="appsettings.json" />
</ItemGroup>
</Project>
@@ -0,0 +1,5 @@
namespace Bts.DataStreamWebApp.Server;
public class CannedQueriesConfig {
public List<CannedQuery>? Items { get; set; }
}
@@ -0,0 +1,6 @@
namespace Bts.DataStreamWebApp.Server;
public class CannedQuery {
public string? Name { get; set; }
public List<string>? Query { get; set; }
}
@@ -0,0 +1,18 @@
namespace Bts.DataStreamWebApp.Server;
using Microsoft.AspNetCore.Mvc;
[ApiController]
[Route("api/[controller]")]
public class CannedQueriesController : ControllerBase {
readonly IConfiguration config;
public CannedQueriesController(IConfiguration config) =>
this.config = config;
[HttpGet(Name = "GetCannedQueries")]
public IEnumerable<CannedQuery>? Get() {
var cannedQueries = config.GetSection("CannedQueries").Get<List<CannedQuery>>()?.ToArray() ?? Array.Empty<CannedQuery>();
return cannedQueries;
}
}
@@ -0,0 +1,22 @@
// using Bts.DataStreamWebApp.Server;
// using Microsoft.AspNetCore.Mvc;
// [ApiController]
// [Route("api/[controller]")]
// public class ConfigController : ControllerBase {
// private readonly IConfiguration _configuration;
// public ConfigController(IConfiguration configuration) {
// _configuration = configuration;
// }
// [HttpGet("websocket")]
// public ActionResult<DataStreamWebSocketConfig> GetWebSocketConfig() {
// var endpoint = $"{_configuration["Kestrel:Endpoints:Http:Url"]}/ws";
// var webSocketConfig = new DataStreamWebSocketConfig {
// Endpoint = endpoint // or Https based on your setup
// };
// return Ok(webSocketConfig);
// }
// }
@@ -0,0 +1,30 @@
namespace Bts.DataStreamWebApp.Server.Controllers;
using Microsoft.AspNetCore.Mvc;
[Route("/ws")]
public class DataStreamSubscriptionController : ControllerBase {
readonly ILogger logger;
public DataStreamSubscriptionController(ILogger<DataStreamSubscriptionController> logger) {
this.logger = logger;
logger.LogInformation("Creating {ControllerType}", GetType().Name);
}
[Route("/ws")]
public async Task Get(CancellationToken cancellationToken) {
if (!HttpContext.WebSockets.IsWebSocketRequest) {
logger.LogError("Expected websocket request on path '{}'", HttpContext.Request.Path);
return;
}
logger.LogInformation("Establishing websocket connection...");
try {
using var webSocket = await HttpContext.WebSockets.AcceptWebSocketAsync();
logger.LogInformation("Websocket connection established.");
using var server = new DataStreamWebSocketServer(webSocket, logger);
await server.Run(cancellationToken);
} catch (Exception ex) {
logger.LogError("DataStreamSubscriptionController.Run threw exception: {Exception}", ex);
}
}
}
@@ -0,0 +1,133 @@
namespace Bts.DataStreamWebApp.Server.Controllers;
using BCG;
using bts.websocket_core.json_rpc;
using Bts.DataStreamCore;
using Bts.DataStreamWebApp.Server.Messages;
using System;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
class DataStreamSubscription : IDisposable, IHostedService {
public readonly ViewStream ViewStream;
public DataPacket? LastFired;
readonly ILogger logger;
readonly WebSocket webSocket;
public DataStreamSubscription(string id, JsonRpcId requestId, ViewStream viewStream, WebSocket webSocket, ILogger logger) {
ViewStream = viewStream;
RequestId = requestId;
Id = id;
this.webSocket = webSocket;
this.logger = logger;
}
public string Id { get; }
public JsonRpcId RequestId { get; }
public SubscriptionStatus Status { get; private set; }
public void Start() {
if (Status == SubscriptionStatus.Subscribed) {
logger.LogWarning("Ignoring redundant call to 'Start'. Id = {Id} Status = {Status}", Id, Status);
return;
}
logger.LogInformation("Subscribing to ViewStream. SubscriptionId = {SubscriptionId}", Id);
Status = SubscriptionStatus.Subscribed;
ViewStream.onChange += onChange;
}
public Task StartAsync(CancellationToken cancellationToken) {
cancellationToken.ThrowIfCancellationRequested();
Start();
return Task.CompletedTask;
}
public Task Clear() {
if (LastFired is null) {
return Task.CompletedTask;
}
logger.LogInformation("Clearing subscription (firing DataPacket to delete all rows). Id = {Id} Status = {Status}", Id, Status);
return sendDataPacket("clear", new(new BlueTable(LastFired.schema), new BlueTable(LastFired.schema), LastFired.newRows));
}
public void Stop() {
if (Status == SubscriptionStatus.Unsubscribed) {
logger.LogWarning("Ignoring redundant call to 'Stop'. Id = {Id} Status = {Status}", Id, Status);
return;
}
logger.LogInformation("Unsubscribing from ViewStream. SubscriptionId = {SubscriptionId}", Id);
Status = SubscriptionStatus.Unsubscribed;
ViewStream.onChange -= onChange;
}
public Task StopAsync(CancellationToken cancellationToken) {
cancellationToken.ThrowIfCancellationRequested();
Stop();
return Task.CompletedTask;
}
public void Dispose() =>
Stop();
async void onChange(object newValue) {
if (newValue is not DataPacket dp) {
logger.LogError("Expected DataPacket in stream callback (actually received {}).", newValue?.GetType());
return;
}
if (webSocket.State != WebSocketState.Open) {
logger.LogError(
"Unsubscribing from stream early because WebSocket state = {WebSocketState}", webSocket.State);
Dispose();
return;
}
await sendDataPacket("subscribe", dp);
}
async Task sendDataPacket(string method, DataPacket dp) {
var serializerOpts = new JsonSerializerOptions {
NumberHandling = JsonNumberHandling.AllowNamedFloatingPointLiterals
};
serializerOpts.Converters.Add(new DataPacketJsonConverter() {
IncludeCurrent = true,
UseObjectRows = true,
});
var res = JsonRpcRequest.CreateNotification(method, new DataStreamSubscriptionUpdate() {
SubscriptionId = Id,
Table = dp
});
var str = serialize(res, serializerOpts);
if (str is null) {
logger.LogError("Unsubscribing from stream early because we failed to serialize an outgoing DataPacket");
Dispose();
return;
}
logger.LogInformation(
"Sending DataPacket. NumRows = {NumRows} NumCols = {NumCols} MessageLength = {MessageLength}",
dp.newRows.Count,
dp.schema.columnNames.Length,
str.Length);
var bytes = Encoding.UTF8.GetBytes(str);
try {
await webSocket.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
LastFired = dp;
} catch (WebSocketException ex) {
logger.LogError($"sendDataPacket failed on webSocket.SendAsync: {ex}");
}
}
string? serialize<T>(T message, JsonSerializerOptions? opts) {
opts ??= JsonSerializerOptions.Default;
try {
return JsonSerializer.Serialize(message, opts);
} catch (Exception ex) {
logger.LogError("Failed to serialize message of type {MessageType}: {Exception}", typeof(T), ex);
}
return null;
}
}
@@ -0,0 +1,13 @@
namespace Bts.DataStreamWebApp.Server;
using BCG;
using System.Text.Json.Serialization;
public class DataStreamSubscriptionUpdate {
[JsonPropertyName("subscription_id")]
public string? SubscriptionId { get; set; }
[JsonPropertyName("table")]
public DataPacket? Table { get; set; }
}
@@ -0,0 +1,89 @@
namespace Bts.DataStreamWebApp.Server;
using Bts.PubSub;
using CommandLine;
using System.Text.Json;
using System.Text.Json.Serialization;
class DataStreamWebAppServerConfig {
/// <summary>
/// The login user for sparkd
/// </summary>
[JsonPropertyName("user")]
[Option("user", HelpText = "The sparkd login user")]
public string? User { get; set; }
/// <summary>
/// The login password for sparkd
/// </summary>
[JsonPropertyName("password")]
[Option("pass", HelpText = "The sparkd login password")]
public string? Password { get; set; }
/// <summary>
/// The config server url for sparkd
/// </summary>
[JsonPropertyName("sparkd")]
[Option("sparkd", HelpText = "The sparkd config server")]
public string? Sparkd { get; set; }
[JsonPropertyName("pubsub")]
public PubSubConfig? PubSub { get; set; }
[Option("content-root-path")]
[JsonPropertyName("content_root_path")]
public string? ContentRootPath { get; set; }
[Option("web-root-path")]
[JsonPropertyName("web_root_path")]
public string? WebRootPath { get; set; }
[Option("environment-name")]
[JsonPropertyName("environment_name")]
public string? EnvironmentName { get; set; }
public DirectoryBrowserOptions? DirectoryBrowser { get; set; }
[Option(
"config",
MetaValue = "PATH",
HelpText = "Path to json file representing these options (DataStreamPublisherConfig)")]
[JsonIgnore]
public string? ConfigFile { get; set; }
public static DataStreamWebAppServerConfig Default() =>
new() {
User = "bts",
PubSub = PubSubConfig.Default(),
};
public void MergeWith(DataStreamWebAppServerConfig other) {
User ??= other.User;
Password ??= other.Password;
PubSub ??= other.PubSub;
Sparkd ??= other.Sparkd;
ContentRootPath ??= other.ContentRootPath;
WebRootPath ??= other.WebRootPath;
ConfigFile ??= other.ConfigFile;
}
/// <summary>
/// Merges options with those defined in `ConfigFile` (does not overwrite
/// non-null existing values).
/// </summary>
/// <exception cref="InvalidOperationException">
/// JsonSerializer.Deserialize returns null
/// </exception>
public void MergeWithConfigFile() {
if (ConfigFile is null) {
return;
}
var cfgString = File.ReadAllText(ConfigFile);
var parsedCfg = JsonSerializer.Deserialize<DataStreamWebAppServerConfig>(cfgString) ??
throw new InvalidOperationException($"Failed to parse json from {ConfigFile}");
MergeWith(parsedCfg);
ConfigFile = null;
}
}
@@ -0,0 +1,5 @@
namespace Bts.DataStreamWebApp.Server;
public class DataStreamWebSocketConfig {
public string? Endpoint { get; set; }
}
@@ -0,0 +1,148 @@
namespace Bts.DataStreamWebApp.Server.Controllers;
using bts.websocket_core.json_rpc;
using Bts.DataStreamWebApp.Server.Messages;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
public class DataStreamWebSocketServer : IDisposable, IAsyncDisposable {
readonly ILogger logger;
readonly WebSocket webSocket;
readonly Dictionary<string, DataStreamSubscription> subscriptions = new();
public DataStreamWebSocketServer(WebSocket webSocket, ILogger logger) {
this.webSocket = webSocket;
this.logger = logger;
}
public async Task<int> Run(CancellationToken cancellationToken) {
logger.LogInformation("DataStreamWebSocketServer running");
await foreach (var req in JsonRpcRequestAsyncEnumerable.Create(webSocket, logger, cancellationToken)) {
await (req.Method switch {
"subscribe" => onSubscribe(req, cancellationToken),
"unsubscribe" => onUnsubscribe(req.Deserialize<UnsubscribeRequest>(), cancellationToken),
_ => Task.CompletedTask,
});
}
logger.LogInformation("DataStreamWebSocketServer stopped");
return 0;
}
/// <summary>
/// Disposes internal subscriptions. To additionally notify the client of the subscription status
/// change(s), use <see cref="DisposeAsync"/>.
/// </summary>
public void Dispose() {
GC.SuppressFinalize(this);
disposeSubscriptions();
}
/// <summary>
/// Disposes internal subscriptions (like <see cref="Dispose"/>), then sends 'subscription status change'
/// notifications to the client (unlike <see cref="Dispose"/>).
/// </summary>
public async ValueTask DisposeAsync() {
GC.SuppressFinalize(this);
var finalSubs = subscriptions.Values.Select(s => (s.RequestId, s.Id)).ToList();
disposeSubscriptions();
// Send a 'subscription status change' msg to the client for each one.
foreach (var (requestId, id) in finalSubs) {
await sendSubscriptionStatus(requestId, id, "unsubscribe", SubscriptionStatus.Unsubscribed, CancellationToken.None);
}
}
async Task onSubscribe(JsonRpcRequest<JsonNode> req, CancellationToken cancellationToken) {
ArgumentNullException.ThrowIfNull(req.Id);
ArgumentException.ThrowIfNullOrEmpty(req.Method);
if (!SubscribeRequest.Parse(req, out var subReq, out var parseErr, out var errCode)) {
logger.LogError("Invalid 'subscribe' request: {Reason}", parseErr);
var rpcError = JsonRpcResponse.CreateError<SubscriptionError>(
req.Id, new() { Code = (long)errCode, Message = "Failed to parse subscription request", Data = parseErr });
await send(rpcError, cancellationToken);
return;
}
var subId = subReq.SubscriptionId ?? throw new InvalidOperationException("SubscriptionId is null (parse false)");
disposeSubscription(subId);
if (!subReq.ParseQuery(out var vs, out parseErr, out errCode)) {
logger.LogError("Invalid 'subscribe' request: {Reason}", parseErr);
var rpcError = JsonRpcResponse.CreateError<SubscriptionError>(
req.Id, new() { Code = (long)errCode, Message = "Failed to parse subscription request", Data = parseErr });
await send(rpcError, cancellationToken);
return;
}
if (subscriptions.TryGetValue(subId, out var oldSub)) {
logger.LogInformation("Disposing existing subscription with id {SubscriptionId}", subId);
oldSub.Dispose();
}
var sub = new DataStreamSubscription(subId, req.Id.Value, vs, webSocket, logger);
subscriptions[subId] = sub;
await sendSubscriptionStatus(req.Id.Value, subId, req.Method, SubscriptionStatus.Subscribed, cancellationToken);
vs.ErrorChanged += OnStreamError;
await sub.StartAsync(cancellationToken);
async void OnStreamError(Exception exception) {
logger.LogError("Error in ViewStream (disposing subscription): {Exception}", exception);
subscriptions.Remove(sub.Id);
sub.Dispose();
var error = new SubscriptionError() {
ErrorMessage = "Error in ViewStream (disposing subscription)"
};
if (exception is not null) {
error.ErrorMessage += ". " + exception.Message;
error.StackTrace = exception.StackTrace?.Split("\n")?.ToList();
}
var rpcError = JsonRpcResponse.CreateError<SubscriptionError>(
req.Id, new() { Code = (long)ErrorCode.ViewStreamError, Message = "Error in ViewStream (disposing subscription)", Data = error });
await send(rpcError, cancellationToken);
}
}
async Task onUnsubscribe(JsonRpcRequest<UnsubscribeRequest> req, CancellationToken cancellationToken) {
ArgumentNullException.ThrowIfNull(req.Id);
ArgumentException.ThrowIfNullOrEmpty(req.Method);
var subId = req.Params?.SubscriptionId ?? string.Empty;
if (!disposeSubscription(subId)) {
logger.LogInformation("Unable to dispose subscription. Subscription with id {Id} not found.", subId);
}
await sendSubscriptionStatus(req.Id.Value, subId, "unsubscribe", SubscriptionStatus.Unsubscribed, cancellationToken);
}
bool disposeSubscription(string subId) {
if (!subscriptions.TryGetValue(subId, out var sub)) {
return false;
}
logger.LogInformation("Disposing subscription with id {Id}", subId);
sub.Dispose();
subscriptions.Remove(subId);
return true;
}
Task sendSubscriptionStatus(JsonRpcId reqId, string subId, string method, SubscriptionStatus status, CancellationToken cancellationToken) {
logger.LogInformation("Sending {Method} ack. SubscriptionId = {SubscriptionId}", method, subId);
var response = JsonRpcResponse.Create(reqId, new SubscriptionStatusMessage() {
SubscriptionId = subId,
Status = status,
});
return send(response, cancellationToken);
}
Task send<T>(T message, CancellationToken cancellationToken) {
if (message is null) {
return Task.CompletedTask;
}
var str = JsonSerializer.Serialize(message);
var bytes = Encoding.UTF8.GetBytes(str);
logger.LogTrace("Sending data. Length = {Length}", bytes.Length);
return webSocket.SendAsync(bytes, WebSocketMessageType.Text, true, cancellationToken);
}
void disposeSubscriptions() {
logger.LogInformation("Disposing DataStreamWebSocketServer subscriptions. Count: {Count}", subscriptions.Count);
subscriptions.Values.DisposeAll();
subscriptions.Clear();
}
}
@@ -0,0 +1,7 @@
FROM bluetradingsystems/linux_dotnet7
RUN dotnet dev-certs https --trust
COPY --chown=bts:bts "build/Release/bin/net7.0/linux-x64/publish" "bin"
COPY --chown=bts:bts "build/Bts.DataStreamWebApp.Client/production/dist" "bin/wwwroot"
COPY --chown=bts:bts "src/DataStreamSuite/Bts.DataStreamWebApp.Server/spark.conf" ".local/share/bts/Bts.DataStreamWebApp.Server/conf/spark.conf"
WORKDIR "/home/bts/bin"
@@ -0,0 +1,5 @@
namespace Bts.DataStreamWebApp.Server;
public class CannedQueriesResponse {
public List<CannedQuery>? Items { get; set; }
}
@@ -0,0 +1,7 @@
namespace Bts.DataStreamWebApp.Server.Messages;
enum ErrorCode {
None = 0,
InvalidSubscriptionRequest,
ViewStreamError,
}
@@ -0,0 +1,72 @@
namespace Bts.DataStreamWebApp.Server.Messages;
using BCG;
using bts.websocket_core.json_rpc;
using Bts.DataStreamExpressions;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Text.Json.Serialization;
public class SubscribeRequest {
[JsonPropertyName("query")]
public string? Query { get; set; }
[JsonPropertyName("subscription_id")]
public string? SubscriptionId { get; set; }
internal static bool Parse(JsonRpcRequest<JsonNode> req, [NotNullWhen(true)]out SubscribeRequest? subscription, [NotNullWhen(false)]out SubscriptionError? error, out ErrorCode errorCode) {
subscription = null;
errorCode = ErrorCode.None;
var paramsJsonObj = req.Params?.AsObject();
error = null;
if (paramsJsonObj is null) {
errorCode = ErrorCode.InvalidSubscriptionRequest;
error = new() { ErrorMessage = "'params' object not found" };
return false;
}
subscription = paramsJsonObj.Deserialize<SubscribeRequest>();
if (subscription is null) {
errorCode = ErrorCode.InvalidSubscriptionRequest;
error = new() { ErrorMessage = "'params' object invalid" };
return false;
}
return true;
}
internal bool ParseQuery([NotNullWhen(true)] out ViewStream? stream, [NotNullWhen(false)] out SubscriptionError? error, out ErrorCode errorCode) {
stream = null;
errorCode = ErrorCode.None;
error = null;
if (Query is null) {
errorCode = ErrorCode.InvalidSubscriptionRequest;
error = new() { ErrorMessage = "'params.query' field not found (expected a string)" };
return false;
}
var exp = DataStreamExpressionsInterpreter.Evaluate(Query);
if (exp is null) {
errorCode = ErrorCode.InvalidSubscriptionRequest;
error = new() { ErrorMessage = "'params.query' expression has invalid syntax" };
return false;
}
Exception? exception = null;
try {
stream = exp.getView(new Dictionary<string, StatementExpression>(0));
} catch (Exception ex) {
exception = ex;
}
if (stream is null) {
errorCode = ErrorCode.InvalidSubscriptionRequest;
error = new() { ErrorMessage = "Failed to create ViewStream from 'params.query' expression (check for typos in arguments/typenames)" };
if (exception is not null) {
error.ErrorMessage += ". " + exception.Message;
error.StackTrace = exception.StackTrace?.Split("\n")?.ToList();
}
return false;
}
return true;
}
}
@@ -0,0 +1,11 @@
namespace Bts.DataStreamWebApp.Server.Messages;
using System.Text.Json.Serialization;
public class SubscriptionError {
[JsonPropertyName("error_message")]
public string? ErrorMessage { get; set; }
[JsonPropertyName("stack_trace")]
public List<string>? StackTrace { get; set; }
}
@@ -0,0 +1,18 @@
namespace Bts.DataStreamWebApp.Server.Messages;
using System.Text.Json.Serialization;
enum SubscriptionStatus {
None,
Subscribed,
Unsubscribed,
}
class SubscriptionStatusMessage {
[JsonPropertyName("subscription_id")]
public string? SubscriptionId { get; set; }
[JsonPropertyName("status")]
[JsonConverter(typeof(JsonStringEnumConverter))]
public SubscriptionStatus Status { get; set; } = SubscriptionStatus.None;
}
@@ -0,0 +1,8 @@
namespace Bts.DataStreamWebApp.Server.Messages;
using System.Text.Json.Serialization;
public class UnsubscribeRequest {
[JsonPropertyName("subscription_id")]
public string? SubscriptionId { get; set; }
}
@@ -0,0 +1,82 @@
using Bts.DataStreamCore.Services;
using Bts.PubSub.Services;
using Spark.Api.Services;
using Spark.Core.Services;
using System.Reactive.Linq;
using System.Reflection;
var builder = WebApplication.CreateBuilder(new WebApplicationOptions {
Args = args,
ApplicationName = "Bts.DataStreamWebApp.Server",
});
ConfigureApp(builder.Configuration, builder.Environment.EnvironmentName);
ConfigureServices(builder.Services, builder.Configuration);
var app = builder.Build();
var wsOpts = new WebSocketOptions {
KeepAliveInterval = TimeSpan.FromMinutes(2)
};
if (app.Environment.IsDevelopment()) {
app.UseDirectoryBrowser("/browse");
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseHttpsRedirection();
app.UseWebSockets(wsOpts);
app.UseDefaultFiles();
app.UseStaticFiles();
app.UseAuthorization();
app.MapControllers();
try {
Console.Write($"Starting Application");
app.Run();
Console.Write($"Application exited gracefully.");
} catch (Exception ex) {
Console.Error.Write($"Application crashed: {ex}");
return 1;
}
return 0;
static void ConfigureApp(ConfigurationManager configBuilder, string environment) {
// We prefix appsettings with the exe name because multiple exes will usually be in the build dir
// (this way, each exe can still load its own appsettings files from the current directory).
var exeName = Assembly.GetCallingAssembly().GetName().Name;
AddJsonFile($"{exeName}.appsettings.json");
AddJsonFile($"{exeName}.appsettings.{environment}.json");
var cfg = configBuilder.GetSection("config")?.Value;
if (!SetBasePath(cfg)) {
AddJsonFile(cfg);
}
bool SetBasePath(string? path) {
if (!Directory.Exists(path)) {
return false;
}
configBuilder.SetBasePath(path);
return true;
}
bool AddJsonFile(string? path) {
if (!File.Exists(path)) {
return false;
}
configBuilder.AddJsonFile(path);
return true;
}
}
static void ConfigureServices(IServiceCollection services, IConfiguration config) {
services.AddDirectoryBrowser()
.AddControllers().Services
.AddEndpointsApiExplorer()
.AddSwaggerGen();
services.AddSpaStaticFiles();
services.AddSpark(config.GetSection("Spark"))
.AddSparkApi(config.GetSection("SparkApi"))
.AddMasterWorker(config.GetSection("MasterWorker"))
.AddViewStreamDatabase()
.AddPubSubClient(config.GetSection("PubSub"));
}
@@ -0,0 +1,29 @@
{
"$schema": "http://json.schemastore.org/launchsettings.json",
"profiles": {
"http": {
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"launchUrl": "swagger",
"applicationUrl": "http://localhost:8080",
"environmentVariables": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
"https": {
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"applicationUrl": "http://localhost:8080;https://localhost:8443",
"environmentVariables": {
"ASPNETCORE_HTTP_PORT": "8080",
"ASPNETCORE_HTTPS_PORT": "8443",
"ASPNETCORE_ENVIRONMENT": "Development",
"ASPNETCORE_HOSTINGSTARTUPASSEMBLIES": "Microsoft.AspNetCore.SpaProxy",
}
},
},
}
@@ -0,0 +1,62 @@
{
"Kestrel": {
"Endpoints": {
"Http": {
"Url": "http://localhost:8080"
},
"Https": {
"Url": "https://localhost:8443"
}
}
},
"PubSub": {
"Routing": {
"PublisherRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://localhost:22019",
"TagRegExp": ".*"
}
],
"SubscriberRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://localhost:22019",
"TagRegExp": ".*"
}
]
}
},
"CannedQueries": [
{
"Name": "BBO",
"Query": [
"bbo"
]
},
{
"Name": "Trades",
"Query": [
"timesales"
]
},
{
"Name": "Instruments",
"Query": [
"instruments"
]
},
{
"Name": "Books",
"Query": [
"books"
]
},
{
"Name": "Benzinga News",
"Query": [
"benzinga_news"
]
}
]
}
@@ -0,0 +1,8 @@
{
"Spark": {
"ConfigServer": "http://sparkd:8000/config"
},
"SparkApi": {
"IsEnabled": true
}
}
@@ -0,0 +1,115 @@
{
"MasterWorker": {
"Interval": "00:00:00.1"
},
"ViewStreamDatabase": {
"Interval": "00:00:00.1"
},
"Spark": {
"ConfigServer": "http://172.30.2.181:8000/config"
},
"SparkApi": {
"Credentials": {
"User": "bts",
"Password": ""
},
"EnableAdmin": true,
"EnableOrders": true,
"EnablePosition": true
},
"PubSub": {
"HeartbeatInterval": 15000,
"MissingHeartbeatsBeforeGc": 50,
"ServerPort": 22019,
"Routing": {
"PublisherRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://172.30.2.181:22019",
"TagRegExp": ".*"
}
],
"SubscriberRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://172.30.2.181:22019",
"TagRegExp": ".*"
}
]
},
"TagTypeToAssemblyMap": {
"Entries": [
{
"HandlerClass": "BCG.PubSubServerTagHandler",
"Assembly": "DataStreamCore.dll"
}
]
},
"TagHandlerFactory": {
"Handlers": [
{
"TagType": "DataStream",
"Assembly": "DataStreamCore",
"Class": "bts.PubsubServerStreamTagHandler"
},
{
"TagType": "System_Sub",
"Assembly": "PubSub",
"Class": "BCG.SystemSubscriptionMapPublisher"
},
{
"TagType": "System_Stats",
"Assembly": "PubSub",
"Class": "BCG.SystemStatsPublisher"
},
{
"TagType": ".*",
"Assembly": "DataStreamCore",
"Class": "bts.PubsubServerGenericDataHandler"
}
]
}
},
"Kestrel": {
"Endpoints": {
"Http": {
"Url": "http://172.30.2.181:8080"
},
"Https": {
"Url": "https://172.30.2.181:8443"
}
}
},
"CannedQueries": [
{
"Name": "BBO",
"Query": [
"bbo"
]
},
{
"Name": "Trades",
"Query": [
"timesales"
]
},
{
"Name": "Instruments",
"Query": [
"instruments"
]
},
{
"Name": "Books",
"Query": [
"books"
]
},
{
"Name": "Benzinga News",
"Query": [
"benzinga_news"
]
}
]
}
@@ -0,0 +1,34 @@
{
"CannedQueries": [
{
"Name": "BBO",
"Query": [
"bbo"
]
},
{
"Name": "Trades",
"Query": [
"timesales"
]
},
{
"Name": "Instruments",
"Query": [
"instruments"
]
},
{
"Name": "Books",
"Query": [
"books"
]
},
{
"Name": "Benzinga News",
"Query": [
"benzinga_news"
]
}
]
}
@@ -0,0 +1,85 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"Kestrel": {
"Endpoints": {
"Http": {
"Url": "http://0.0.0.0:8080"
},
"Https": {
"Url": "https://0.0.0.0:8443"
}
}
},
"MasterWorker": {
"Interval": "00:00:00.1"
},
"ViewStreamDatabase": {
"Interval": "00:00:00.1"
},
"SparkApi": {
"IsEnabled": false,
"Credentials": {
"User": "bts",
"Password": ""
}
},
"PubSub": {
"HeartbeatInterval": 15000,
"MissingHeartbeatsBeforeGc": 50,
"ServerPort": 22019,
"Routing": {
"PublisherRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://pubsub:22019",
"TagRegExp": ".*"
}
],
"SubscriberRoutes": [
{
"Name": "default",
"ServerAddress": "bts-cloud://pubsub:22019",
"TagRegExp": ".*"
}
]
},
"TagTypeToAssemblyMap": {
"Entries": [
{
"HandlerClass": "BCG.PubSubServerTagHandler",
"Assembly": "DataStreamCore.dll"
}
]
},
"TagHandlerFactory": {
"Handlers": [
{
"TagType": "DataStream",
"Assembly": "DataStreamCore",
"Class": "bts.PubsubServerStreamTagHandler"
},
{
"TagType": "System_Sub",
"Assembly": "PubSub",
"Class": "BCG.SystemSubscriptionMapPublisher"
},
{
"TagType": "System_Stats",
"Assembly": "PubSub",
"Class": "BCG.SystemStatsPublisher"
},
{
"TagType": ".*",
"Assembly": "DataStreamCore",
"Class": "bts.PubsubServerGenericDataHandler"
}
]
}
}
}
@@ -0,0 +1,6 @@
FROM bluetradingsystems/linux_dotnet7
RUN dotnet dev-certs https --trust
COPY --chown=bts:bts "bin" "bin"
COPY --chown=bts:bts "webapp.spark.conf" ".local/share/DataStream/conf/spark.conf"
WORKDIR "/home/bts/bin"
@@ -0,0 +1,15 @@
Logging {
value {
console {
default_level: "Info"
}
syslog {
default_level: "Info"
}
file {
default_level: "Info"
}
initial_queue_size: 120001
max_queue_growth: 4
}
}
@@ -0,0 +1,168 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title></title>
<style>
table {
border: 0
}
.commslog-data {
font-family: Consolas, Courier New, Courier, monospace;
}
.commslog-server {
background-color: red;
color: white
}
.commslog-client {
background-color: green;
color: white
}
</style>
</head>
<body>
<h1>ALT WebSocket Sample Application</h1>
<p id="stateLabel">Ready to connect...</p>
<div>
<label for="connectionUrl">WebSocket Server URL:</label>
<input id="connectionUrl" />
<button id="connectButton" type="submit">Connect</button>
</div>
<p></p>
<div>
<label for="sendMessage">Message to send:</label>
<input id="sendMessage" disabled />
<button id="sendButton" type="submit" disabled>Send</button>
<button id="closeButton" disabled>Close Socket</button>
</div>
<h2>Communication Log</h2>
<table style="width: 800px">
<thead>
<tr>
<td style="width: 100px">From</td>
<td style="width: 100px">To</td>
<td>Data</td>
</tr>
</thead>
<tbody id="commsLog">
</tbody>
</table>
<script>
var connectionUrl = document.getElementById("connectionUrl");
var connectButton = document.getElementById("connectButton");
var stateLabel = document.getElementById("stateLabel");
var sendMessage = document.getElementById("sendMessage");
var sendButton = document.getElementById("sendButton");
var commsLog = document.getElementById("commsLog");
var closeButton = document.getElementById("closeButton");
var socket;
var scheme = document.location.protocol === "https:" ? "wss" : "ws";
var port = document.location.port ? (":" + document.location.port) : "";
connectionUrl.value = scheme + "://" + document.location.hostname + port + "/ws" ;
function updateState() {
function disable() {
sendMessage.disabled = true;
sendButton.disabled = true;
closeButton.disabled = true;
}
function enable() {
sendMessage.disabled = false;
sendButton.disabled = false;
closeButton.disabled = false;
}
connectionUrl.disabled = true;
connectButton.disabled = true;
if (!socket) {
disable();
} else {
switch (socket.readyState) {
case WebSocket.CLOSED:
stateLabel.innerHTML = "Closed";
disable();
connectionUrl.disabled = false;
connectButton.disabled = false;
break;
case WebSocket.CLOSING:
stateLabel.innerHTML = "Closing...";
disable();
break;
case WebSocket.CONNECTING:
stateLabel.innerHTML = "Connecting...";
disable();
break;
case WebSocket.OPEN:
stateLabel.innerHTML = "Open";
enable();
break;
default:
stateLabel.innerHTML = "Unknown WebSocket State: " + htmlEscape(socket.readyState);
disable();
break;
}
}
}
closeButton.onclick = function () {
if (!socket || socket.readyState !== WebSocket.OPEN) {
alert("socket not connected");
}
socket.close(1000, "Closing from client");
};
sendButton.onclick = function () {
if (!socket || socket.readyState !== WebSocket.OPEN) {
alert("socket not connected");
}
var data = sendMessage.value;
socket.send(data);
commsLog.innerHTML += '<tr>' +
'<td class="commslog-client">Client</td>' +
'<td class="commslog-server">Server</td>' +
'<td class="commslog-data">' + htmlEscape(data) + '</td></tr>';
};
connectButton.onclick = function() {
stateLabel.innerHTML = "Connecting";
socket = new WebSocket(connectionUrl.value);
socket.onopen = function (event) {
updateState();
commsLog.innerHTML += '<tr>' +
'<td colspan="3" class="commslog-data">Connection opened</td>' +
'</tr>';
};
socket.onclose = function (event) {
updateState();
commsLog.innerHTML += '<tr>' +
'<td colspan="3" class="commslog-data">Connection closed. Code: ' + htmlEscape(event.code) + '. Reason: ' + htmlEscape(event.reason) + '</td>' +
'</tr>';
};
socket.onerror = updateState;
socket.onmessage = function (event) {
commsLog.innerHTML += '<tr>' +
'<td class="commslog-server">Server</td>' +
'<td class="commslog-client">Client</td>' +
'<td class="commslog-data">' + htmlEscape(event.data) + '</td></tr>';
};
};
function htmlEscape(str) {
return str.toString()
.replace(/&/g, '&amp;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;');
}
</script>
</body>
</html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

@@ -0,0 +1,176 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title></title>
<style>
table {
border: 0
}
.commslog-data {
font-family: Consolas, Courier New, Courier, monospace;
}
.commslog-server {
background-color: red;
color: white
}
.commslog-client {
background-color: green;
color: white
}
</style>
</head>
<body>
<h1>DataStreams via WebSocket</h1>
<p id="stateLabel">Ready to connect...</p>
<div>
<label for="connectionUrl">WebSocket Server URL:</label>
<input id="connectionUrl" />
<button id="connectButton" type="submit">Connect</button>
</div>
<p></p>
<div>
<label for="sendMessage">Query:</label>
<input id="sendMessage" disabled />
<button id="sendButton" type="submit" disabled>Send</button>
<button id="closeButton" disabled>Close Socket</button>
</div>
<h2>Communication Log</h2>
<table style="width: 800px">
<thead>
<tr>
<td style="width: 100px">From</td>
<td style="width: 100px">To</td>
<td>Data</td>
</tr>
</thead>
<tbody id="commsLog">
</tbody>
</table>
<script>
var connectionUrl = document.getElementById("connectionUrl");
var connectButton = document.getElementById("connectButton");
var stateLabel = document.getElementById("stateLabel");
var sendMessage = document.getElementById("sendMessage");
var sendButton = document.getElementById("sendButton");
var commsLog = document.getElementById("commsLog");
var closeButton = document.getElementById("closeButton");
var socket;
var scheme = document.location.protocol === "https:" ? "wss" : "ws";
var port = document.location.port ? (":" + document.location.port) : "";
connectionUrl.value = scheme + "://" + document.location.hostname + port + "/ws" ;
function updateState() {
function disable() {
sendMessage.disabled = true;
sendButton.disabled = true;
closeButton.disabled = true;
}
function enable() {
sendMessage.disabled = false;
sendButton.disabled = false;
closeButton.disabled = false;
}
connectionUrl.disabled = true;
connectButton.disabled = true;
if (!socket) {
disable();
} else {
switch (socket.readyState) {
case WebSocket.CLOSED:
stateLabel.innerHTML = "Closed";
disable();
connectionUrl.disabled = false;
connectButton.disabled = false;
break;
case WebSocket.CLOSING:
stateLabel.innerHTML = "Closing...";
disable();
break;
case WebSocket.CONNECTING:
stateLabel.innerHTML = "Connecting...";
disable();
break;
case WebSocket.OPEN:
stateLabel.innerHTML = "Open";
enable();
break;
default:
stateLabel.innerHTML = "Unknown WebSocket State: " + htmlEscape(socket.readyState);
disable();
break;
}
}
}
closeButton.onclick = function () {
if (!socket || socket.readyState !== WebSocket.OPEN) {
alert("socket not connected");
}
socket.close(1000, "Closing from client");
};
sendButton.onclick = function () {
if (!socket || socket.readyState !== WebSocket.OPEN) {
alert("socket not connected");
}
const rpcReq = {
jsonrpc: "2.0",
method: "subscribe",
params: {
query: sendMessage.value,
},
id: 1,
};
const rpcReqStr = JSON.stringify(rpcReq);
socket.send(rpcReqStr);
commsLog.innerHTML += '<tr>' +
'<td class="commslog-client">Client</td>' +
'<td class="commslog-server">Server</td>' +
'<td class="commslog-data">' + htmlEscape(rpcReqStr) + '</td></tr>';
};
connectButton.onclick = function() {
stateLabel.innerHTML = "Connecting";
socket = new WebSocket(connectionUrl.value);
socket.onopen = function (event) {
updateState();
commsLog.innerHTML += '<tr>' +
'<td colspan="3" class="commslog-data">Connection opened</td>' +
'</tr>';
};
socket.onclose = function (event) {
updateState();
commsLog.innerHTML += '<tr>' +
'<td colspan="3" class="commslog-data">Connection closed. Code: ' + htmlEscape(event.code) + '. Reason: ' + htmlEscape(event.reason) + '</td>' +
'</tr>';
};
socket.onerror = updateState;
socket.onmessage = function (event) {
commsLog.innerHTML += '<tr>' +
'<td class="commslog-server">Server</td>' +
'<td class="commslog-client">Client</td>' +
'<td class="commslog-data">' + htmlEscape(event.data) + '</td></tr>';
};
};
function htmlEscape(str) {
return str.toString()
.replace(/&/g, '&amp;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;');
}
</script>
</body>
</html>
@@ -0,0 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Hosting" Version="7.0.1" />
</ItemGroup>
</Project>
@@ -0,0 +1,59 @@
namespace Bts.Extensions.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using System.Diagnostics;
public static class HostBuilderExtensions {
public static IHostBuilder CreateBtsBuilder(string[] args) {
// Obtain calling assembly name here, rather than in the lambda where it is actually needed,
// because want the name of the assembly that called this function, not the assembly that will eventually invoke
// the lambda...that will be the Microsoft.Extensions.Hosting assembly.
// Get the name of the assembly that called this function. That will act as the "appsettings"
// json file prefix
var exeName = GetCallingAssemblyName();
return Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((context, configBuilder) =>
ConfigureApp(configBuilder, context.HostingEnvironment.EnvironmentName, exeName));
}
public static void ConfigureApp(IConfigurationBuilder configBuilder, string? environment = null, string? prefix = null) {
var baseFileName = prefix is null ? "appsettings" : $"{prefix}.appsettings";
AddJsonFile($"{baseFileName}.json");
if (environment is not null) {
AddJsonFile($"{baseFileName}.{environment}.json");
}
var cfg = configBuilder.Build().GetSection("config")?.Value;
if (!SetBasePath(cfg)) {
AddJsonFile(cfg);
}
bool SetBasePath(string? path) {
if (!Directory.Exists(path)) {
return false;
}
configBuilder.SetBasePath(path);
return true;
}
bool AddJsonFile(string? path) {
if (!File.Exists(path)) {
return false;
}
configBuilder.AddJsonFile(path);
return true;
}
}
public static string GetCallingAssemblyName() =>
new StackTrace()
.GetFrame(2)
?.GetMethod()
?.Module
?.Assembly
?.GetName()
?.Name
?? throw new InvalidOperationException("Failed to determine name of calling assembly (calling method is null)");
}
@@ -0,0 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net7.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>
Loaded 100 of 2349 files, more files were not shown because too many files have changed in this diff. Show more