Commit Graph
13278 Commits
Author SHA1 Message Date
John Colvin 17e60df32b cleanup: Remove unused file PubSub3Config.cs 2024-06-06 09:15:16 -05:00
John Colvin c99d5fe24c Add support for core PubSub config via json 2024-06-06 09:15:16 -05:00
John Colvin 6049826e90 build: Make build_data_stream.py cross platform
Previously it only worked on Windows
2024-06-06 09:15:16 -05:00
John Colvin 672149dd89 Add more descriptive logs (via stdout) during initialization
The C++ configure/initialize routines were particularly opaque
2024-06-06 09:15:16 -05:00
John Colvin 8c99147f04 build: Build cmake presets in distinct binary dirs
It may use more disk space, but we really don't have a choice. CMake
projects with different parameters, preprocessors, source files, etc.
are not designed to share binary dirs. For example, the libspark.so file
that the Spark front-end is supposed to use is not necessarily the same
as the libspark.so file the Api Server is supposed use.
2024-06-06 09:15:16 -05:00
John Colvin 4d4b0b7ae3 build: Give api server distinct shm prefix 2024-06-06 09:15:16 -05:00
John Colvin 1a7955c1b1 build: Remove unnecessary cache var/macro for api server
* BUILD_API_SERVER (cmake cache var)
* SPARK_ENABLE_API_SERVER (preprocessor defined conditionally w/ BUILD_API_SERVER)

They were originally added to control whether or not MD Consumer extern
fns were built. They have been made redundant by the more appropriately
named EXPORT_MD_CONSUMER cache var/macro (both have same name now).
2024-06-06 09:15:16 -05:00
John Colvin f8fc2a1d96 cleanup: Remove static service Locator class
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
2024-06-06 09:15:16 -05:00
John Colvin 9a7b3cb77c refactor: Reduce code (simplify) in BTS.init
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
2024-06-06 09:15:16 -05:00
John Colvin 8a3e4edb4d Dispose connections in SimpleTCPServer.Dispose 2024-06-06 09:15:15 -05:00
John Colvin e59183bb1e improvement: Allow app data dir to be overriden w/ env var
Historically, this has been hardcoded to ~/sparkdata on Linux, and
%APPDATA%/bts on Windows.
2024-06-06 09:15:15 -05:00
John Colvin dd4d4e8dc6 fix: Expand path vars in cmake presets at runtime
The binaries should be portable to other machines (paths shouldn't
be fixed to the user's home dir at compile time).
2024-06-06 09:15:15 -05:00
John Colvin f1744450ca build: Add script to build api server 2024-06-06 09:15:15 -05:00
John Colvin 3a7827a75b build: Automatically load vs dev prompt for win builds 2024-06-06 09:15:15 -05:00
John Colvin 8d22e91c0d fix build: Remove references to HighPerformanceTimer
An old custom timer class from Edge that we no longer need. We're
replacing it w/ a dotnet builtin timer class called 'Stopwatch'.
2024-06-06 09:15:15 -05:00
John Colvin c425145fc9 Remove mongo stream from MarketDataPublisher.json
It doesn't work unless you have mongodb running (a bit of a burden when you
just want to publish market data)
2024-06-06 09:15:15 -05:00
John Colvin 4855935ae0 improvement: Get PubSubRoutes from config (json)
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin 65f412ec9f fix: Fix myLoadAllTypes impl
* Fix lowercase typo in hardcoded assemblies (case matters now that we
  run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
  and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin 40681c6093 Add MaybeNullWhen(false) attribute to TryGetFirst result
So callers can assume non-null when it returns true (for IEnumerables of
non-null ref types)
2024-06-06 09:15:15 -05:00
John Colvin ab2d36e523 vscode: Add tasks.json
This provides a good base of operations to build/launch our now numerous
projects
2024-06-06 09:15:09 -05:00
John Colvin b1ab3a66d2 build: Fix old build_sparkd.py script
Things have become a lot simpler since it was originally written.
2024-06-05 15:44:36 +00:00
John Colvin 5e7e7e8ef4 test: Add FilterStreamTests 2024-06-05 15:44:35 +00:00
John Colvin 9291d30ae9 build: Add cmake opt to export MDConsumerInterop fns
Now that we have all these different builds (sparkd, spark, api server,
data streams) - many of which are cross platform - we need a proper
toggle for this (relying on 'IF WIN32' preprocessors doesn't cut it)
2024-06-05 15:44:35 +00:00
John Colvin 549708fcc2 fix: Use ".dll" file ext when loading assemblies by path 2024-06-05 15:44:35 +00:00
John Colvin 1d5551f301 build: Add cmake preset to build data streams on linux 2024-06-05 15:44:35 +00:00
John Colvin 283db0f9ea build: Define shm prefix and app data dir in CMakePresets
It's more organized this way, with the input variables for each
configuration defined in one place.
2024-06-05 15:44:35 +00:00
John Colvin 9fb04b2db9 cleanup: Remove unused cmake preset 'spark-api-server-base'
This was a hidden preset designed for common Linux/Windows config.
Neither Linux nor Windows configs were inheriting it.
2024-06-05 15:44:35 +00:00
John Colvin ceca96e10a Add build_data_stream.sh script for Linux 2024-06-05 15:44:35 +00:00
John Colvin fc48cc64af fix build (webapp): Add --include=dev arg to npm install
The apt repo finally updated npm from 8.X.X to 10.8.0. This broke the
Spark.Admin.WebApp build because the new version doesn't install dev
dependencies by default.
2024-06-05 15:44:35 +00:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin e39108bbda fix build (sparkd): Fix typo
'git' was appended to a type name. I suspect a keyboard focus conflict
between vs code editor/terminal panes.
2024-06-03 13:46:51 +00:00
John Colvin 6240d89a08 Map PubSubServerConfig json to historic PubSub3Config 2024-06-03 13:46:51 +00:00
John Colvin c76700e5e5 cleanup: Remove unused PubSubRoute array from DSQA 2024-06-03 13:46:51 +00:00
John Colvin a8a0e8dea0 cleanup: Remove unused PubSubConfigureForm 2024-06-03 13:46:51 +00:00
John Colvin d65a2d6ed6 test: Add DataPacketBuilderTests
This tests somes facilities used in the new MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d3eb29087b refactor: Impl IDictionary<int, IBlueRow> in BlueTable 2024-06-03 13:46:51 +00:00
John Colvin d6fae238f9 Add IObservable ViewStream wrapper 2024-06-03 13:46:51 +00:00
John Colvin 44b46033e3 cleanup: Remove HighPerformanceTimer 2024-06-03 13:46:51 +00:00
John Colvin 5741ec9ab2 cleanup: Remove unnecessary usings (namespaces) 2024-06-03 13:46:51 +00:00
John Colvin 596975b309 style: Apply modern formatting to Crono 2024-06-03 13:46:51 +00:00
John Colvin 9050318ceb refactor: Replace HighPerformanceTimer w/ Stopwatch
This is needed to build DataStreamSuite on Linux.

'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
John Colvin 7ee6933789 refactor: Add formal config classes for prepublish streams
They can be passed to command line via --config <json_file>
2024-06-03 13:46:51 +00:00
John Colvin 306c572b53 improvement: Polish clis in DataStreamSuite
Specifically PubSubServer, DataStreamPublisher and DSQA

* Use CommandLineParser (nuget pkg)
* Clean up csproj files a bit while we're at it
2024-06-03 13:46:51 +00:00
John Colvin 63306173b1 refactor: Reorganize NumericComparer, support all types
Get rid of redundant UniversalNumericComparer while we're at it
2024-05-25 15:37:33 +00:00
John Colvin df1b45ca3e cleanup: Remove leftover legacy files 2024-05-25 15:37:33 +00:00
John Colvin c31a1f1bf3 cleanup: Switch assert statements to 'constraint model'
This gets rid of a few more build warnings in DataStreamSuite
2024-05-25 15:37:33 +00:00
John Colvin e95997ea17 cleanup: Chip away a few nullability warnings in DataStreamSuite
This is a start, but there're still > 700 of them!
2024-05-25 15:37:33 +00:00
John Colvin 12a070fd8d build: Remove DSQA from DataStreamSuite.sln to its own sln
So we can build DataStreamSuite.sln on Linux (DSQA depends on WinForms)
2024-05-25 15:37:33 +00:00
John Colvin e0459c54fd build: Fix 'pubsub' -> 'PubSub' typo in csproj ProjectReferences
Now all DataStream projects build on Linux (except DSQA, which is a winforms app)
2024-05-25 15:37:33 +00:00