Commit Graph
12553 Commits
Author SHA1 Message Date
John Colvin 6c2bfee616 fix (docker): Set default user without specific group
Bugs (fixed):
  * The default user does not have sudo permission in the container,
    unless the user is specified explicitly in the 'docker exec' cmd
    (via -u <user>).

    Cause: When the default user is specified with a specific group
    (in this case, its namesake 'bts'), it is excluded from all other
    groups, including 'sudo'.
2023-04-03 11:43:37 -04:00
John Colvin 30c5ea7fb2 chore: Name devcontainer scripts after associated json prop 2023-04-03 10:49:58 -04:00
John Colvin a6831b5548 chore: Add devcontainer.json to bootstrap devcontainers 2023-04-03 10:49:58 -04:00
John Colvin b42943fe8d chore: Rename Docker image 'spark-dev' -> 'sparkd-dev'
This is a more accurate and appropriate name than 'spark-dev' because
the image is designed to build/run Sparkd and the Spark Api Server (both
of which are daemons, hence the appropriateness of the 'd' suffix). It
isn't designed to build/run the front-end desktop application, Spark.
2023-04-03 10:28:51 -04:00
John Colvin 06c833e96c fix: Use correct prefix in md cleanup
Bugs (fixed):
  * Assert crash in MDSource.h "Marketdata heartbeats are bad..."

  Cause: The MD cleanup procedure, which is supposed to delete md files
  from shared memory (from previous runs), wasn't deleting anything.
  The file globbing pattern it was using (to determine which files to
  delete) never produced any matches because it had the wrong prefix. As
  a result, market data would often be loaded with a stale heartbeat
  timestamp from a previous run.

  Note: This occurred as a side effect of making the shared memory prefix
  configurable when we ported the api server over to Linux (so the api
  server and sparkd could run on the same container/machine whilst
  maintaining independent shared memory files).
2023-04-03 10:28:51 -04:00
John Colvin 9aff02b225 chore: Remove unused docker scripts 2023-04-03 10:28:51 -04:00
John Colvin 4229a523a8 docker: Init devcontainer bashrc and config.fish 2023-04-03 10:28:51 -04:00
John Colvin 82e9e6b6bf docker: Change default docker_user to "bts"
This way we can all use the same dev image. It also means one less
variable in our containers that paths/scripts need to account for.
VS Code can automatically update the remote user's uid in the container
to avoid permission issues in mounted dirs.
2023-04-03 13:48:43 +00:00
John Colvin 904a6db1d0 fix: Add Mercury default conf
The quickbus was essentially broken when running on the 'default'
defaults, for lack of a better word, because the ingoing/outgoing queues
in BrokerProxy would be initialized with size 0.
2023-04-03 13:48:16 +00:00
John Colvin 2a9f03741e chore: Move docker files into main repo
Changes to Docker-related files/configuration don't occur in a vacuum.
They are made in coordination with changes to the main repository, so it
doesn't make sense to track them separately.

Note: Removing the 'windows' Docker files completely. No one has has
used them in a long, long time. The client build would be broken in
there anyways, the image is based on .NET Framework 3.5!
2023-04-03 13:48:16 +00:00
John Colvin fd3de056c1 style (cli): Clean up 'key-code' cmd
* Rename 'key-code' -> 'key' (better grammar for the sub cmds)
* Rename 'key-gen' -> 'gen'

Now the 'key' cmds are 'sparkd key gen/encrypt/decrypt'
2023-03-31 05:00:07 +00:00
John Colvin 978a1e8566 dev: Integrate Docker w/ VS Code devcontainers
Hopefully this resolves all of the trouble/confusion in setting up
sparkd Docker containers
2023-03-31 04:59:58 +00:00
John Colvin 0d23fd50f5 improvement: Indent protos in sparkd config cmd
So proto values are indented relative to the key, which is the type name
2023-03-28 10:45:03 -04:00
John Colvin 367c9c1c7c chore: Include metrics.h in MDSource.h
This file was broken, as it defines methods that depend on metrics.h.
However, the build wasn't broken because references to such methods
are in files that happen to include metrics.h themselves.
2023-03-28 00:45:07 -04:00
John Colvin 78f5526ce7 refactor: Add spark_test, Spark.Console to Spark.Api.Server sln 2023-03-28 00:45:07 -04:00
John Colvin a1d7ebd260 improvement: Lower 'finit_spark_lib' log level to Diag 2023-03-28 00:45:07 -04:00
John Colvin 3fda1cb369 improvement: Add null check for rule_desc in cfg lookup
It is possible rule_desc is null given an inappropriate field_name arg
2023-03-28 00:45:07 -04:00
John Colvin 2c2a68022b improvement: Spruce up 'config' cli
It's naturally a bit messy because we expose the c++ cli to csharp, so
it can be invoked via Spark.Console (cs) or sparkd
2023-03-28 00:45:07 -04:00
John Colvin 45c35ba1b2 fix build: Fix install path issues in cmake 2023-03-26 13:44:25 -04:00
John Colvin 77c61d5e3f fix build: Run dotnet tool restore when building webapp 2023-03-26 13:44:25 -04:00
John Colvin 12962ed5ca Update docker/linux, and externals
* Fixes a few build dep issues
* Adds [docker] user in container, rather than image
* Automatically installs emumba libs when creating/starting new docker
  containers
* Binds sparkd ports to the host so containers don't need to share the
  host's network stack (allows us to run sparkd in containers in WSL)
* Adds flag to start_docker.sh to allow mounting existing dirs
* Improves start_docker.sh help text
2023-03-26 13:44:22 -04:00
John Colvin f472db002c feat (sparkd cli): Bring back 'web' cmd 2023-03-25 11:46:23 -04:00
John Colvin 6939f32f69 refactor: Rename 'web' -> 'Admin.WebApp' in CMakeLists 2023-03-25 11:43:29 -04:00
John Colvin 0107d934e5 fix: Resolve req paths correctly in Admin WebApp 2023-03-25 11:00:45 -04:00
Taha Afzal ce9880a9b1 changed samples 2023-03-21 12:44:43 -04:00
Taha Afzal 317f17d7da added examples 2023-03-21 12:39:58 -04:00
Taha Afzal 7da128da6b removed allow all headers 2023-03-21 01:16:02 -04:00
Taha Afzal f603866860 changed config to make on titan 2023-03-21 01:14:56 -04:00
John Colvin dc2d525955 fix build: Reorder link libraries
For some reason, it the previous order of link libs caused an unresolved
external symbol related to open ssl (looks like a version macro)
2023-03-20 13:26:44 -04:00
John Colvin e97335189a refactor: Rename Spark.Admin.Console -> WebApp
This is a more appropriate name (it is a web app after all), especially
now that we've renamed sparkcmd -> Spark.Console (a literal console
app). The old name implies a parallel between the two apps that doesn't
exist.
2023-03-20 13:26:44 -04:00
John Colvin 81ce58268f improvement: Clean up Spark.Console
* Reword cli verbs, opts, and help text
* Rename files, folders, and namespaces
* Run code formatter
2023-03-20 13:26:44 -04:00
John Colvin 7dace3fc1d refactor: Rename sparkcmd -> Spark.Console
This is a more conventional name for a companion console app
2023-03-20 13:26:44 -04:00
John Colvin f4794bea33 improvement: Add sparkd (config-server) opt to configcheck cli 2023-03-20 13:26:44 -04:00
John Colvin b044ff3b2a chore: Transcribe more proto2 defaults to .conf files
After upgrading to proto3
2023-03-20 13:26:44 -04:00
John Colvin cd7856c46e fix build (spark): Add DeploymentType proto to spark_core 2023-03-20 13:26:44 -04:00
John Colvin 3be0ee6072 refactor: Move LazyObservable to Spark.Core.Reactive 2023-03-20 13:26:44 -04:00
John Colvin 198d26fda1 refactor: Move ObservableExtensions to Spark.Core.Reactive 2023-03-20 13:26:44 -04:00
John Colvin 8b26ac4456 chore: Remove unused code 2023-03-20 13:26:44 -04:00
John Colvin d6f172e953 refactor: Move KeyedValueSet to Spark.Core.Collections
Also rename to 'KeyedValueCollection'
2023-03-20 13:26:44 -04:00
John Colvin 91ee9b739a refactor: Move ChangeDictionary to Spark.Core.Collections 2023-03-20 13:26:44 -04:00
John Colvin b568128180 improvement: Set sys cfg build_type to release (if not debug)
For completeness sake
2023-03-20 13:26:44 -04:00
John Colvin 53a87530ee refactor: Remove erroneous 'qualifier' field from SystemConfig 2023-03-20 13:26:44 -04:00
John Colvin 450de301a9 build: Generate tsconfig file in the build dir
It never should have been dumped in the source directory in the first
place.
2023-03-20 13:26:44 -04:00
John Colvin 462b0f4eda build: Get rid of redundant py CMake utils 2023-03-20 13:26:44 -04:00
John Colvin 17402ce94a refactor: Replace os::bts_data_path w/ SystemConfig::data_path 2023-03-20 13:26:44 -04:00
John Colvin b7fae8a6f5 fix: Init quickbus node cfg w/ proper defaults 2023-03-20 13:26:44 -04:00
John Colvin 4058a98290 refactor: Rename .conf files w/ snake_case
To be consistent w/ corresponding .proto files
2023-03-20 13:26:44 -04:00
John Colvin 06d7693ed6 fix: Set hardcoded defaults for quickbus cfg
Remove these hardcoded vals once config defaults have been properly
refactored
2023-03-20 13:26:44 -04:00
John Colvin 2b01f3d902 fix: Ensure shared mem block size isn't 0 2023-03-20 13:26:44 -04:00
John Colvin d231ee50b7 chore: Fill in tedious defaults for jetfix cfg 2023-03-20 13:26:43 -04:00