John Colvin
9930b65ddc
refactor: Split MarketDataGateway up into providers/sources
...
MarketDataGateway still exists, but is greatly simplified. It creates
several `IMarketDataProvider`s, each of which generates a list of
`IMarketDataSource`s. The market data sources provide market data to an
`IMarketDataListener`.
* EmumbaMarketDataProvider
* CCApiMarketDataProvider
* PyMarketDataProvider
* OnixsMarketDataProvider
* ReplayMarketDataProvider
As a side effect, individual py market data sources are now explicitly
limited to a single venue (it probably should have been this way all
along, since the shared memory writers do not expect updates to involve
multiple venues).
2025-05-20 20:20:07 +00:00
arham
bc5ddade7f
Add: okx exe small changes to maybe make ping better
2025-05-01 09:42:54 +00:00
John Colvin
b4f5e66466
fix: Set okx perp exp to year 2100
...
Because the year 3000 is too big for signed int64
nanosecond timestamp
2025-04-29 19:35:15 +00:00
John Colvin
69a0bcc9cd
fix: Request binance fut md on ccapi dispatcher thread
2025-04-29 19:32:21 +00:00
John Colvin
d72ebdf421
perf: Send Binance Usds Futs md rest reqs on background thread
2025-04-29 19:32:21 +00:00
John Colvin
8e93856767
fix: Send exchangeInfo req via our HttpClient instead of ccapi
...
For binance usds futs (we already did this for spots)
Bugs (fixed):
* We get a boost beast exception about the read limit being exceeded
when requesting securities from Binance prod url (testnet works).
2025-04-29 19:32:21 +00:00
John Colvin
c2150ae450
refactor: Move BinanceUsdsFuturesFilterTuple fns to cpp file
2025-04-29 19:32:21 +00:00
John Colvin
3756a8a8b7
Add api key type enum to binance exe gateway config
...
ED25519 keys offer performance benefits over the HMAC-SHA256
keys when using the websocket api, as you can log in on the websocket
connection to avoid signing each request.
2025-04-29 19:32:21 +00:00
John Colvin
61f04c4605
fix: Implement state machine for binance cxl/replace
2025-04-28 17:12:59 +00:00
John Colvin
ab217401eb
cleanup: Move CreateOrderMsg impl to .cpp, get rid of unused code
2025-04-28 17:12:58 +00:00
John Colvin
40ad1f1b48
Update binance websocket endpoints
...
The api just changed, and old urls will be retired in May 2025
2025-04-28 17:12:58 +00:00
John Colvin
484e36b330
refactor: Make timestamp internal value to signed (int64)
...
I am sick of casting uint64 -> int64. The latter is a much more
common argument type than the former.
2025-04-28 17:12:58 +00:00
John Colvin
5476466d55
Fetch secrets for config profiles with new aws sso config
2025-04-28 17:12:58 +00:00
John Colvin
29bf7d8d7e
deploy (fix): Update okx urls
...
Their aws domain has been retired.
2025-04-28 17:12:58 +00:00
John Colvin
98bc78463d
build (linux): Migrate catch2 dep to vcpkg (from externals)
2025-04-28 17:12:58 +00:00
John Colvin
2a17d656a5
fix: Dump json to str in LOG statements
...
The nlohmann::json obj isn't formattable (i.e., doesn't have a
fmt::formatter<T> specialization)
2025-04-28 17:12:58 +00:00
John Colvin
623012b0b0
deploy: Add "sparkd_symbols" image to Dockerfile
...
Normally, we bake spark config into the image, which is simpler and less
error prone than managing separate configurations per host machine
and/or container. The sparkd_symbols deployment breaks the mold. Config
is mounted to a dir on the host machine so symbols can be added/removed
via config (i.,e., spark.conf) when we bounce sparkd.
2025-04-17 05:53:05 +00:00
arham
960a8af77a
Fix: OKX futures working now
2025-04-10 05:06:17 +00:00
arham
14e3eee0bc
Add: config changes to make server controller work and set up hk zen server
2025-04-09 09:42:47 -04:00
John Colvin
7c7c689170
fix build: Don't cast to unsigned int when checking bytesRead < 0
2025-04-04 15:36:55 -04:00
John Colvin
116506c8da
fix build: Lock databento version to v0.31.0
...
Previously, we were simply fetching HEAD. This was fine until they
released v0.32.0 (on 04/02/2025), which introduced a new dependency
requirement that broke our build: OpenSSL >= 3.0.
2025-04-04 15:36:55 -04:00
Adeel Khan
3feda49ef1
Fix: Taifex Time Conversion
2025-04-04 15:36:54 -04:00
Adeel Khan
10318be7bc
FIX: Taifex Crash Fixed, Added Diag Logs
2025-04-04 15:36:54 -04:00
Adeel Khan
f7d525a527
ADD: TBBO & Symbol wise Naming in Recorder
2025-04-04 15:36:54 -04:00
Adeel Khan
d2fe9f167a
ADD: TWSE Code Complete
2025-04-04 15:36:54 -04:00
Adeel Khan
d575ba0ea8
ADD: TWSE throughout Sparkd
2025-04-04 15:36:54 -04:00
John Colvin
e888149c41
Add order msg throttle to sparkd config profiles
2025-03-20 17:29:43 +00:00
John Colvin
d49efef93d
Increase ram to 8gb in alphaworx_prod config
...
The previous value of 4gb wasn't enough, as the container would
occasionally crash with a SIGKILL interrupt.
2025-03-19 15:26:14 +00:00
John Colvin
3980f2d030
Add order msg throttle to alphaworx_prod config
2025-03-17 13:00:15 +00:00
John Colvin
6d4d8f6ac1
fix: Disable ccapi order book cross check for Binance md
...
When enabled, the checks fail regularly (usually more than once per
hour). Sometimes many of them happen in rapid succession before sparkd
crashes. I'm disabling them in the hopes that the crosses are benign,
and this will stop the crashes.
2025-03-17 00:51:54 +00:00
John Colvin
379a46afa2
Add order msg throttle to sim_dev config
2025-03-15 20:07:14 +00:00
John Colvin
b251e1d47e
build: Generate semantic version header from git
...
Dump it to console and log file when sparkd runs
2025-03-14 16:00:06 +00:00
Mohammad Arham
5ba7901258
Fix: undo special case for btc swap and add ccp for swaps
2025-03-12 20:47:44 +00:00
Mohammad Arham
bc6eb59110
Fix: Change cttype from optional to string
2025-03-12 20:45:27 +00:00
Mohammad Arham
ff86b10895
Fix: temp fix to move okx btc swap to config for min tick
2025-03-11 20:35:24 +00:00
John Colvin
d0abca3fd9
Fix symbols and account alphaworx_prod config
2025-03-11 19:47:34 +00:00
John Colvin
7895fe7357
Remove taifex and glbx from NotSimExecution in conf profiles
2025-03-11 15:24:08 +00:00
John Colvin
2072105c0f
cleanup: Add using aliases to make code more concise
2025-03-10 14:53:07 +00:00
John Colvin
d663c86aa0
improvement: Only load synth metadata from disk once in md dumper
...
Instead of every time a synth is dumped (i.e., do it lazily, outside the
loop body)
2025-03-10 14:53:07 +00:00
John Colvin
4a2dfa46a1
improvement: Print delimiter line when dumping secs to console
2025-03-10 14:53:07 +00:00
John Colvin
6e35db6a24
improvement: Apply filter to shared mem secs in md dumper
...
Previously, filter only applied to db securities (i.e., when --db flag
was used)
2025-03-10 14:52:39 +00:00
John Colvin
a1912907b8
Add brief readmes for sparkd config profiles
2025-03-10 05:24:49 +00:00
John Colvin
b1b3dcdeb8
Add sim_dev config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
96f28acf91
Add dev config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
c2c2658e21
Add alphaworx_prod config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
d6b8e359cd
Expand environment vars in binance gw config
2025-03-10 03:05:59 +00:00
John Colvin
84a0850d95
Tweak binance_dev config profile
...
* Factor out repeated symbol list into SymbolLists entry
* Change "binance_dev" account to "BINANCE_DEV" in exe gw json
* Adjust log level
* Remove unused config:
- ExchangeSubAccounts
- JetFIX_Connection.max_order_msgs_per_window
2025-03-10 03:05:59 +00:00
John Colvin
b9fe39f6e3
refactor: Make MDRecorder join the run thread in destructor
...
This wasn't broken, but detached threads bother me.
2025-03-10 03:05:59 +00:00
John Colvin
b555850015
build: Package & deploy sparkd with docker compose
2025-03-10 03:05:59 +00:00
Mohammad Arham
93b93969f5
Add: config to control databento md better
2025-03-06 02:44:40 +00:00