Commit Graph
13678 Commits
Author SHA1 Message Date
John Colvin 47cc33afc7 refactor: Move TradeRecord into lib_domain 2024-11-29 15:05:11 +00:00
John Colvin ffb1d9fd47 refactor: Move TradingStatus enum to lib_domain 2024-11-29 15:05:11 +00:00
John Colvin 340e91f164 build: Update webpack-cli in admin webapp 2024-11-29 15:05:11 +00:00
John Colvin 3fb0b420db build: Configure admin webapp build entirely in CMakeLists
Get rid of build_admin_web_app.py
2024-11-29 15:05:11 +00:00
John Colvin 7cc938d85e refactor: Move domain types into new lib_domain
Also rename files in snake_case (standard naming convention for protos)
https://protobuf.dev/programming-guides/style/
2024-11-29 15:05:11 +00:00
John Colvin f20e013e4e refactor: Move collections into lib_system 2024-11-29 15:05:11 +00:00
John Colvin bf9a6c3431 refactor: Move a handful of types/utils to lib_system 2024-11-29 15:05:11 +00:00
John Colvin f052c591eb build: Add preset to build with 6 jobs (-j 6)
Otherwise the devcontainer may hog too much cpu and cause the container to freeze
2024-11-29 15:05:11 +00:00
John Colvin 583cf0bb16 cleanup: Remove unused includes
Mostly collection macro headers
2024-11-29 15:05:11 +00:00
John Colvin bb72e1b8ad cleanup (build): Move protobuf service generation cmake into lib_spark
Because lib_spark owns all the services
2024-11-29 15:05:11 +00:00
John Colvin 5466058ef2 refactor: Rename protobuf out dir "include" -> "protobufs"
This is a more accurate and precise name. The dir contains c++ code generated
from .proto files - both sources (.cc) and headers (.pb.h).
2024-11-29 15:05:11 +00:00
John Colvin e98bb730e4 refactor: Move ValueWrapper into lib_system
So other projects can use it
2024-11-29 15:05:10 +00:00
John Colvin 1f9a67f912 Include bts_assert.h in bts_fatal.h
We shouldn't have to include bts_assert.h when we're already including bts_fatal.h
2024-11-29 15:05:10 +00:00
John Colvin ec0712b0cc refactor: Split MarketDataGateway up into providers/sources
MarketDataGateway still exists, but is greatly simplified. It creates
several `IMarketDataProvider`s, each of which generates a list of
`IMarketDataSource`s. The market data sources provide market data to an
`IMarketDataListener`.

* EmumbaMarketDataProvider
* CCApiMarketDataProvider
* PyMarketDataProvider
* OnixsMarketDataProvider
* ReplayMarketDataProvider

As a side effect, individual py market data sources are now explicitly
limited to a single venue (it probably should have been this way all
along, since the shared memory writers do not expect updates to involve
multiple venues).
2024-11-29 15:05:10 +00:00
John Colvin 6ea7774e5f cleanup: Remove superfluous code 2024-11-29 15:05:10 +00:00
John Colvin f2f2024562 improvement: Set SHARED_MEMORY_PREFIX to an environment variable
So we can effectively define it at runtime. We don't want to have to
produce different builds just to change the shared mem prefix.
2024-11-29 15:05:10 +00:00
John Colvin 25ea9fac1d fix: Always prepend proc name to shared mem prefix
Our shared memory model isn't really designed to be shared across processes.
Maybe some day it will.
2024-11-29 15:05:10 +00:00
John Colvin 98a6a357bd cleanup: Remove redundant macro def 2024-11-29 15:05:10 +00:00
John Colvin 62a4073748 Skip md recording thread loop if no recorders configured to run 2024-11-29 15:05:10 +00:00
John Colvin bad698a4ff Handle signal interrupts explicitly in sparkd
Otherwise the python interpretter, which is instantiated when we load
python md plugins, prevents the application from exiting.
2024-11-29 15:05:10 +00:00
John Colvin 7ed2019006 feat: Add support for python md plugins 2024-11-29 15:05:10 +00:00
John Colvin fd2fdf7d39 fix build: Ensure py scripts use unix line endings
I'm not sure why, but my sparkd build (linux) suddenly began failing to
generate cpp headers for protos. The error message indicated it was due
to an unexpected carriage return (\r) in plugin.py:

/usr/bin/env: ‘python\r’: No such file or directory
2024-11-29 15:05:10 +00:00
John Colvin 55349b2791 refactor: Add lib_system proj for common utils
Migrates the following:
* Delegate.h
* bts_array_size.h
* string_utils.h

Note: File names in the new lib are snake_case (this seems to be the
standard naming convention for c++).
2024-11-29 15:05:10 +00:00
John Colvin 5d7aa63869 cleanup: Trivial whitespace cleanup in sparkd CMakeLists 2024-11-29 15:05:10 +00:00
John Colvin 81057cfe77 build: Add cmake fns to symlink and install local libs
The code needed to do this (inline) is ugly and verbose
2024-11-29 15:05:10 +00:00
John Colvin 54bd95e8b2 cleanup: Use conventional UPPER_CASE var names in sparkd CMakeLists 2024-11-29 15:05:10 +00:00
John Colvin 841d1be1dd cleanup: Trivial whitespace cleanup in Protobuf.cmake 2024-11-29 15:05:10 +00:00
John Colvin d54d5b31ad build: Clean up data_files cmake proj (support --target clean) 2024-11-29 15:05:10 +00:00
John Colvin 457e619261 refactor: Organize libspark proj w/ stuttering include
This new structure provides several benefits:

* Allows us to install multiple CMake projects in the same destination
  dir (without include path conflicts).
  - This makes shipping the trading algo sdk a trivial excercise. Simply
    copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
  modules, which are easier to manage.
2024-11-29 15:05:10 +00:00
John Colvin fcace19a5a vscode: Add debug cfg to skip sparkd build 2024-11-29 15:00:58 +00:00
John Colvin e815362b50 fix build: Add ShmSegmentName.cpp to windows client sources 2024-11-29 15:00:58 +00:00
John Colvin 1c3238b86d Make a few timestamp fns const
They should have been const in the first place
2024-11-29 15:00:58 +00:00
John Colvin 092a29d28b Add config profiles to libspark
This is part of a larger effort to unify the libspark build, and eliminate
compile time differences between the builds used by our various services
2024-11-29 15:00:58 +00:00
John Colvin 6202ba52b4 Add api server to docker compose 2024-11-29 15:00:58 +00:00
John Colvin 8afac9e4ac Rename pingtest "sparkd-ping-test", make it depend on sparkd
Add it to a "ping-test" profile, as we may add similar tests for other services
2024-11-29 14:58:54 +00:00
John Colvin 5602c6c642 cleanup: Remove unnecessary env vars from webapp docker compose 2024-11-29 14:58:54 +00:00
John Colvin be77b630dd vscode: Fix sparkd debug config 2024-11-29 14:58:54 +00:00
John Colvin b20c38bd1a build: Resolve app data dir and shared mem prefix at runtime
We really can't define them at compile time anymore as we build multiple
exes that depend on libspark in the same dir (e.g., spark api server,
pubsub, ds-publisher, webapp, etc.). That means one libspark.so (or dll)
must serve all exes. We should compile a consistent libspark to mitigate
the risk of running an exe with the wrong version (of libspark). For
example, running pubsub with a libspark configured for spark api server,
leftover from an earlier build.
2024-11-29 14:58:54 +00:00
John Colvin 932918f473 Don't log "initializing spark config" if it has already been initialized 2024-11-29 14:58:54 +00:00
John Colvin 24cbfd9062 vscode: Add debug cfg for spark api server 2024-11-29 14:58:54 +00:00
John Colvin e39de508f7 Use dependency injection in spark api server 2024-11-29 14:58:54 +00:00
John Colvin a9542b3dd1 Add SparkLoggerAdapter for Msft ILogger 2024-11-29 14:57:44 +00:00
John Colvin badb5e4c2d Display exception stack trace w/ 1 call per line in ds webapp 2024-11-29 14:57:44 +00:00
John Colvin 7d870d925b Remove unused ErrorView from ds webapp 2024-11-29 14:57:44 +00:00
John Colvin a47fc311b3 Propagate ViewStream errors to webapp frontend 2024-11-29 14:57:44 +00:00
John Colvin 4e63428540 Add ErrorChanged event to propagate DataStream exceptions 2024-11-29 14:57:44 +00:00
John Colvin 29b305635d Set columns to empty array rather than null in ds webapp
If no data has been received. Since the 'columns' property was marked
'required', this eliminates a warning that was being logged to console.
2024-11-29 14:57:44 +00:00
John Colvin ef10ae6075 Improve alignment/spacing in ds webapp 2024-11-29 14:57:44 +00:00
John Colvin 7e8ee6a22c Configure datastream deployment with docker-compose
To deploy to aws, connect to our aws vpn then run:
tools/deploy-docker-compose.sh

The deploy script does the following:
* builds sparkd
* builds datastream apps (pubsub, md-publisher, and webapp)
* uploads the build to aws
* bounces all services (sparkd, pubsub, md-publisher, and webapp)

Note: the name 'md-publisher' refers to DataStreamPublisher w/ MarketData
configuration
2024-11-29 14:57:44 +00:00
John Colvin 95d3f643d5 Remove "BTS Cloud Streams" title from webapp
Space is precious when viewing on phone (tbh, it didn't look that good
anyways).
2024-11-29 14:57:44 +00:00