Commit Graph
13691 Commits
Author SHA1 Message Date
John Colvin 9a9bcef348 Add ostream operator for Venue 2024-10-30 19:35:05 +00:00
John Colvin 8502b113fe Add ostream operator for SecurityType 2024-10-30 19:35:05 +00:00
John Colvin 979b2ccb6f refactor: Normalize sec key indexing for new (domain) md sources
This is part of a larger effort to unify the way we index securities in order
to generate unique ids. Older md sources (e.g., MDP3Listener) use
SecurityIDRecorder to generate ids, which uses a different database and
different key-string formatting than the IDGenerator used by the newer md
sources (e.g., GeneralMarketDataListener and various ccapi-based sources).
2024-10-30 19:35:05 +00:00
John Colvin 6e12c5b895 refactor: Replace ConfigDownloader w/ lib_http fns
Essentially all it did was fetch config files via http GET. I ported some http
fns over from the cme-websocket-connector repo, which uses curlpp instead of
curl, and also supports async requests (coroutines) via the 'concurrencpp'

https://github.com/David-Haim/concurrencpp
2024-10-30 19:35:06 +00:00
John Colvin 2624a2c00d cleanup: Get rid of another Security type aliases 2024-10-30 19:35:06 +00:00
John Colvin fb01953296 cleanup: Remove unused code in Security.h/cpp 2024-10-30 19:35:06 +00:00
John Colvin 72a3358f44 Fix typo in 'Quantity' FMT_IMPL
I didn't realize the "value()" function rounded out the decimal,
so I replaced it with a call to "full_value()".
2024-10-30 19:35:06 +00:00
John Colvin 2a505e8ce5 diag: Log when sqlite db is created 2024-10-30 19:35:06 +00:00
John Colvin a4d4a938b0 diag: Log when an MDProducer is created 2024-10-30 19:35:06 +00:00
John Colvin c2922d5e02 diag: Add logs around synthetic security discovery in MDConsumer 2024-10-30 19:35:06 +00:00
John Colvin 8e109c6450 refactor: Break shared mem md code up into smaller files
For easier reading/navigation
2024-10-30 19:35:06 +00:00
John Colvin 85b1909543 refactor: Break md relay/query server code up into smaller files
To make code navigation easier. This was a difficulty which I recently
confronted, while investigating a bug that prevented synthetics from
showing up the ui.
2024-10-30 19:35:06 +00:00
John Colvin ba4e20552e refactor: Get rid of 'TimeStamp' typedef in MDStructure.h
Before this commit, we had 4 different TimeStamp types/identifiers, which is
confusing enough as it is:

* bts::core::TimeStamp
  - in system/time_stamp.h
* bts::utils::TimeStamp
  - in spark/utils/TimeStamp.h
* bts::TimeStamp
  - in system/bts_chrono.h
  - alias for std::chrono::system_clock::time_point
* bts::md::TimeStamp
  - in spark/md/Shm/MDStructure.h
  - alias for bts::utils::TimeStamp
  - removed in this commit

Additional aliases for these TimeStamps only adds to the confusion (esp
when the alias comes from a nested include).
2024-10-30 19:35:06 +00:00
John Colvin b1f1987922 Make security 'formattable'
So it can be used as a param in LOG statements
2024-10-30 19:35:06 +00:00
John Colvin b13725bc09 refactor: Change refs/raw ptrs to smart ptrs in md gateway 2024-10-30 19:35:06 +00:00
John Colvin 85dbee2f00 Fix typo in market_data.conf 2024-10-30 19:35:06 +00:00
John Colvin 14232f5644 Add BTC and ETH to VENUE_SIM in symbols.conf 2024-10-30 19:35:06 +00:00
John Colvin 89d49cd721 fix: Resolve main logging dir in new SparkDataDirectory class 2024-10-30 19:35:05 +00:00
John Colvin 590bc1749a refactor: Replace if/else with switch cases 2024-10-30 19:35:05 +00:00
John Colvin 339767a51f fix: Resolve onixs log & data dirs relative to app data root
When our logging code moved to lib_system, it lost access to the config
system. As a result, the "log_path" function, which was used by ICEMain
and MDP3Main, broke. I added a new class, SparkDataDirectory, to be the
central/common authority on paths to local storage.
2024-10-30 19:35:05 +00:00
John Colvin 47f2f16cd3 cleanup: Remove over-engineered subcommands
We don't need explicit subcommands to 'run onixs' md (just use config).
2024-10-30 19:35:05 +00:00
John Colvin d1b28a86c8 fix: Ensure IDGenerator instance is created before deref 2024-10-30 19:35:05 +00:00
John Colvin 50e7ad925e fix: Expand undefined vars to empty string
Bugs (fixed):
  * Md shared memory file names have unexpanded variable: '${BTS_SHARED_MEMORY_PREFIX}'
2024-10-30 19:35:05 +00:00
John Colvin 132201d2cf fix: Emplace MDConsumerHandle instead of copy assign 2024-10-30 19:35:05 +00:00
John Colvin 5543725ee2 Generate c++ protos w/ relative paths
We shouldn't be generating protos defined outside of the current CMakeLists.txt dir.
2024-10-30 19:35:05 +00:00
John Colvin 9c535616df fix build: Make venue conversion constexpr 2024-10-30 19:35:05 +00:00
John Colvin 1dd2db41f2 build: Use new/robust init_library fn in lib_domain 2024-10-30 19:35:05 +00:00
John Colvin a451c3e880 refactor: Move config services to lib_system
The proto files are still in lib_spark, w/ the exception of
system_config.proto, since that is a dep of ConfigurationState.
2024-10-30 19:35:05 +00:00
John Colvin 5485cc9c4f refactor: Move logging into lib_system
This is a fundamental part of our application suite, so system seems
like a sensible home.

Also moved a few logging deps (to lib_system):
* bts_chrono.h
* bts_no_copy.h
* defer.h
* dispatcher.h/cpp
* file_utils.h/cpp
* type_name.h
2024-10-30 19:35:05 +00:00
John Colvin 5e7e20df5a build: Move http code into its own lib 2024-10-30 19:35:05 +00:00
John Colvin 28f5181585 refactor: Move thread_manager to lib_system 2024-10-30 19:35:05 +00:00
John Colvin 5e7a478bf5 build (linux): Migrate catch2 dep to vcpkg (from externals) 2024-10-30 19:35:05 +00:00
John Colvin 0abc386d71 build (linux): Migrate fmt dep to vcpkg (from externals)
Now both linux and windows use vcpkg's fmt
2024-10-30 19:35:05 +00:00
John Colvin 53b040342a cleanup: Remove unnecessary includes 2024-10-30 19:31:41 +00:00
John Colvin e85f669534 refactor: Move threads.proto to lib_system 2024-10-30 19:31:41 +00:00
John Colvin 935d414b7f refactor: Move warn_ctrl.h into lib_system
This is another logging dep
2024-10-30 19:31:41 +00:00
John Colvin 89f4fe88f9 build: Link boost_filesystem in lib_system 2024-10-30 19:31:41 +00:00
John Colvin 161e0a8c3b refactor: Move os.h/cpp and deps to lib_system
This is a prerequisite for the next objective: move logging into its own lib.
As logging is a core dependency for so much of our system, this will enable
us to move more complex services (like md providers) into smaller libs.
2024-10-30 19:31:41 +00:00
John Colvin f5a6dc7a88 cleanup: Remove unused code 2024-10-30 19:31:41 +00:00
John Colvin 298a2f0617 cleanup: Remove more unnecessary includes 2024-10-30 19:31:41 +00:00
John Colvin c6fc1a0f89 refactor: Move SecurityMarketDataStatistics to lib_domain 2024-10-30 19:31:41 +00:00
John Colvin 6cd7baf903 cleanup: Remove unnecessary code 2024-10-30 19:31:41 +00:00
John Colvin 7d7621d832 cleanup: Remove empty header FTXSecurity.h 2024-10-30 19:31:41 +00:00
John Colvin ef8e4c4998 cleanup: Remove temporary redirect headers
These were used to make code migration smoother (to the new lib_system and lib_domain)

* spark/algorithm/TryCatch.h -> system/algorithm/try_catch.h
* spark/core/BoolName.h -> system/bool_name.h
* spark/core/EnumMacros.h -> system/enum_macros.h
* spark/core/FMT.h -> system/FMT.h
* spark/core/FMT_IMPL.h -> system/FMT_IMPL.h
* spark/core/TimeSpan.h -> system/time_span.h
* spark/core/TimeStamp.h -> system/time_stamp.h
* spark/domain/Price.h -> domain/price.h
* spark/io/JsonReader.h -> system/json/json_reader.h
* spark/io/JsonDumper.h -> system/json/json_dumper.h
2024-10-30 19:31:41 +00:00
John Colvin ad2e631bbc refactor: Move RFQ to lib_domain 2024-10-30 19:31:41 +00:00
John Colvin dccc9f0099 refactor: Move AggressorSide proto to lib_domain
Get rid of non-proto AggressorSide enum while we're at it (consolidate to 1 type)
2024-10-30 19:31:41 +00:00
John Colvin bab5eb4c70 cleanup: Remove more unused code 2024-10-30 19:31:41 +00:00
John Colvin e0608c3791 refactor: Move a few Order types to lib_domain 2024-10-30 19:31:41 +00:00
John Colvin e69c4782b5 refactor: Move TimeInForce to lib_domain 2024-10-30 19:31:41 +00:00
John Colvin 7ab045b120 cleanup: Remove unused code 2024-10-30 19:31:41 +00:00