Commit Graph
426 Commits
Author SHA1 Message Date
John Colvin 603f810465 build: Configure admin webapp build entirely in CMakeLists
Get rid of build_admin_web_app.py
2024-10-17 05:33:11 +00:00
John Colvin 304c6fd4d1 refactor: Move domain types into new lib_domain
Also rename files in snake_case (standard naming convention for protos)
https://protobuf.dev/programming-guides/style/
2024-10-17 05:33:11 +00:00
John Colvin 7e79430aeb refactor: Organize libspark proj w/ stuttering include
This new structure provides several benefits:

* Allows us to install multiple CMake projects in the same destination
  dir (without include path conflicts).
  - This makes shipping the trading algo sdk a trivial excercise. Simply
    copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
  modules, which are easier to manage.
2024-10-16 18:35:15 +00:00
John Colvin 95ba1bb0bd Add api server to docker compose 2024-10-14 17:21:49 +00:00
John Colvin b0bc68997b Configure datastream deployment with docker-compose
To deploy to aws, connect to our aws vpn then run:
tools/deploy-docker-compose.sh

The deploy script does the following:
* builds sparkd
* builds datastream apps (pubsub, md-publisher, and webapp)
* uploads the build to aws
* bounces all services (sparkd, pubsub, md-publisher, and webapp)

Note: the name 'md-publisher' refers to DataStreamPublisher w/ MarketData
configuration
2024-10-14 17:21:49 +00:00
John Colvin 0e50994c2d build: Change publish_data_stream.py to publish the sln
We were publishing individual projects as a workaround for the
dependency issue (bug) that caused some projects to incorrectly target
net8.0. The fact that publishing individual projects "just worked" was
probably dumb luck. Now that the underlying issue has been addressed, we
can resume publishing the sln as a whole.
2024-10-14 17:21:46 +00:00
John Colvin 2c813da889 fix: Add self-signed certificate to Dockerfile for ds webapp 2024-10-14 17:21:46 +00:00
John Colvin b4eaba8af4 Define webapp specific env vars in generic datastream app deploy script
For now, this makes things easier, as I don't need to write a special script just for the webapp
2024-10-14 17:21:46 +00:00
John Colvin 7801ac9300 Add config profile and Dockerfile args to deploy script 2024-10-14 17:21:46 +00:00
John Colvin 4f6614c080 Copy DataStream WebApp 'dist' to 'wwwroot' in deploy script 2024-10-14 17:21:46 +00:00
John Colvin 36f9e29520 Add scripts to deploy data stream apps 2024-10-14 17:21:46 +00:00
John Colvin b966352d79 Publish Bts.DataStreamWebApp.Server along with PubSub & DataStream 2024-10-14 17:21:46 +00:00
John Colvin 7b57a07a31 build: Add scripts to publish data stream apps 2024-10-14 17:21:45 +00:00
John Colvin dfa47bd88e Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-10-14 17:21:45 +00:00
John Colvin 5a51d6e7f6 build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-10-14 17:21:45 +00:00
John Colvin fac11fdfe2 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-10-14 17:21:45 +00:00
John Colvin 79f3431e77 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-10-14 17:21:45 +00:00
John Colvin b69b7b3350 vscode: Use '\' dir separator in show_last_run.cmd
This script only runs on Windows
2024-06-08 15:08:25 -05:00
John Colvin f93d28ee76 vscode: Add task to view logs for last DSQA run 2024-06-07 11:34:36 -05:00
John Colvin 7b5a6031af build: Use consistent naming conventions in CMake presets
We'll use similar naming conventions for our (vscode) tasks
2024-06-06 09:15:17 -05:00
John Colvin 54e96a06cb build: Build DSQA on Windows in build_data_stream.py
It was only building DataStreamSuite.sln before, which is crossplatform
now so it no longer includes DSQA.
2024-06-06 09:15:16 -05:00
John Colvin 6049826e90 build: Make build_data_stream.py cross platform
Previously it only worked on Windows
2024-06-06 09:15:16 -05:00
John Colvin f1744450ca build: Add script to build api server 2024-06-06 09:15:15 -05:00
John Colvin 3a7827a75b build: Automatically load vs dev prompt for win builds 2024-06-06 09:15:15 -05:00
John Colvin b1ab3a66d2 build: Fix old build_sparkd.py script
Things have become a lot simpler since it was originally written.
2024-06-05 15:44:36 +00:00
John Colvin ceca96e10a Add build_data_stream.sh script for Linux 2024-06-05 15:44:35 +00:00
John Colvin df1b45ca3e cleanup: Remove leftover legacy files 2024-05-25 15:37:33 +00:00
John Colvin 2d8047ba80 build: Tidy up build/publish_spark scripts
In preparation for the first deployment of the "development" branch.
2024-04-02 01:47:47 -05:00
John Colvin a091a95f50 build: Add .cmd wrapper script for build_data_stream.py 2023-10-02 19:38:38 -04:00
John Colvin fd9eaa65ed build: Add script to build DataStreamSuite and cpp deps 2023-09-30 13:54:58 -04:00
John Colvin cc2959249e build: Add lexer/parser.cs files to src ctrl
Move all lexer/parser files into a single dir, DataStreamGrammar, for
organizational purposes and to avoid dependency conflicts when
generating code. Note that we'll only need to regen cs if we modify the
parser/lexer files, which seems unlikely.
2023-09-03 13:53:07 -04:00
John Colvin e1b6971667 build: Greatly simplify build_spark.py
Make it a thin wrapper around dotnet/cmake clis
2023-08-23 00:46:21 -04:00
John Colvin cf34c13f03 build: Replace build_api_server.py w/ cmake/dotnet clis
Add instructions to build the Api Server in README
2023-08-22 14:38:39 -04:00
John Colvin 7545d3fd5c Merge remote-tracking branch 'origin/spark_crypto.prod' into development 2023-08-04 13:40:04 -04:00
John Colvin 2ed23159ca style: Clean up jenkins build script
* Get rid of double backslashes in quoted paths to improve readability
* Call 'activate.bat' instead of 'activate' (official python docs say to
  use the former)
* Add docs
2023-07-26 15:26:30 -04:00
John Colvin faa8438adf build: Make jenkins build script independent of working dir
This alleviates an unneccessary burden on callers (previously, it had to
be run from the repo root dir).
2023-07-26 11:59:03 -04:00
John Colvin 9e4354fb55 fix buid: Don't goto ERROR just bc venv already exists
This seems like a typo
2023-07-18 23:40:18 -04:00
John Colvin e6bb4c73a8 fix build (jenkins): Upgrade pip via 'python -m pip'
For some reason 'pip install --upgrade pip' works on my laptop, but fails on
the Jenkins box.
2023-07-18 22:53:32 -04:00
John Colvin e43a2a4818 build: Add build script for new Jenkins (on aws) 2023-07-18 22:25:56 -04:00
John Colvin cde0f44b7d build: Default to Release cfg for 'publish' target 2023-07-18 22:25:06 -04:00
John Colvin 5230d6bab5 build: Clean up build_spark.py 2023-06-23 15:33:40 +00:00
John Colvin e3cd910acd build: Use CMakePresets in build_spark.py 2023-06-21 11:07:45 -04:00
John Colvin cb47177b8e build: Remove unused 'link_build_dir' arg 2023-06-21 11:07:45 -04:00
John Colvin af0b7d9146 fix build: Set cmake policy CMP0076 before target_sources
* Removes warning when configuring build on Windows
* Moves 'vs_filters.cmake' to 'cmake' dir (w/ other cmake scripts)
2023-06-21 11:07:45 -04:00
John Colvin e1234fe583 fix: Fix typo in log helper script 'show_last_run.sh'
So it actually applies the 'num_lines' override arg
2023-06-07 13:16:21 -04:00
John Colvin e64dc49a64 fix build: Respect cfg path order in gen'd defaults
The path order represents the value-inheritance hierarchy. This way,
values from files earlier in the path take precedence over those from
later files.
2023-06-07 13:16:21 -04:00
John Colvin 82c126362d fix build: Fix a few path errors in publish target 2023-05-31 22:13:52 -04:00
arhamGIT 3dd7d0999e merge 2023-04-26 12:33:32 -04:00
John Colvin c85e221e9f build: Automatically run build_spark.py in venv 2023-04-20 00:46:23 -04:00
John Colvin 947c071cee build: Remove unused cmake def 2023-04-20 00:46:22 -04:00