This is a prerequisite to synchronizing security ids stored via IDGenerator
(the "new" way to store ids, added when we implemented Deribit), and
SecurityDB (the original way from spark days). The problem was that these
2 implementations were using separate databases. We need to unify the
interfaces between these storage services to avoid breaking existing code.
This is part of a larger effort to unify the way we index securities in order
to generate unique ids. Older md sources (e.g., MDP3Listener) use
SecurityIDRecorder to generate ids, which uses a different database and
different key-string formatting than the IDGenerator used by the newer md
sources (e.g., GeneralMarketDataListener and various ccapi-based sources).
Essentially all it did was fetch config files via http GET. I ported some http
fns over from the cme-websocket-connector repo, which uses curlpp instead of
curl, and also supports async requests (coroutines) via the 'concurrencpp'
https://github.com/David-Haim/concurrencpp
To make code navigation easier. This was a difficulty which I recently
confronted, while investigating a bug that prevented synthetics from
showing up the ui.
Before this commit, we had 4 different TimeStamp types/identifiers, which is
confusing enough as it is:
* bts::core::TimeStamp
- in system/time_stamp.h
* bts::utils::TimeStamp
- in spark/utils/TimeStamp.h
* bts::TimeStamp
- in system/bts_chrono.h
- alias for std::chrono::system_clock::time_point
* bts::md::TimeStamp
- in spark/md/Shm/MDStructure.h
- alias for bts::utils::TimeStamp
- removed in this commit
Additional aliases for these TimeStamps only adds to the confusion (esp
when the alias comes from a nested include).
When our logging code moved to lib_system, it lost access to the config
system. As a result, the "log_path" function, which was used by ICEMain
and MDP3Main, broke. I added a new class, SparkDataDirectory, to be the
central/common authority on paths to local storage.
This is a fundamental part of our application suite, so system seems
like a sensible home.
Also moved a few logging deps (to lib_system):
* bts_chrono.h
* bts_no_copy.h
* defer.h
* dispatcher.h/cpp
* file_utils.h/cpp
* type_name.h
This is a prerequisite for the next objective: move logging into its own lib.
As logging is a core dependency for so much of our system, this will enable
us to move more complex services (like md providers) into smaller libs.