fix: Ensure md-publisher waits for pubsub and sparkd

Bugs (fixed):
  * md-publisher silently fails if it tries to connect before those services are "ready".

The Docker compose "depends_on" property ensures sparkd and pubsub
containers are up, but doesn't prevent startup race bugs (e.g., if
md-publisher and pubsub start around the same time, and md-publisher
tries to connect before pubsub has started accepting connections). To
address these race conditions, we use a nifty tool called "dockerize"

References:
  * https://github.com/jwilder/dockerize
This commit is contained in:
John Colvin committed 2024-07-29 15:54:26 +00:00
1 parent f59751b6e1
commit f80c876df8
2 files changed
+23 -2

No files matched your search

+15 -2
View File
@@ -16,8 +16,21 @@ services:
md-publisher:
build:
dockerfile: src/DataStreamSuite/Dockerfile
command: [ "./DataStreamPublisher" ]
dockerfile: src/DataStreamSuite/DataStreamPublisher/Dockerfile
depends_on:
- pubsub
- sparkd
command:
[
"dockerize",
"-wait",
"tcp://pubsub:22019",
"-wait",
"tcp://sparkd:8000",
"-timeout",
"10s",
"./DataStreamPublisher"
]
environment:
- DOTNET_ENVIRONMENT=MarketData
@@ -1,5 +1,13 @@
FROM bluetradingsystems/linux_dotnet7
ENV DOCKERIZE_VERSION v0.7.0
USER root
RUN apt-get update \
&& apt-get install -y wget \
&& wget -O - https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz | tar xzf - -C /usr/local/bin \
&& apt-get autoremove -yqq --purge wget && rm -rf /var/lib/apt/lists/*
USER bts
COPY --chown=bts:bts "build/Release/bin/net7.0/linux-x64/publish" "bin"
COPY --chown=bts:bts "src/DataStreamSuite/DataStreamPublisher/spark.conf" ".local/share/DataStream/conf/spark.conf"
WORKDIR "/home/bts/bin"