refactor: Add DataStream service modules for Dependency Injection

Namely, in DataStreamPublisher and PubSubServer. This is part of a broader
paradigm shift towards Dependency Injection and Factory Pattern.

https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection-guidelines

* Use interfaces to decouple services:
  - Decouple ManagedSubscriber from ISubscriberCore impl
  - Decouple PubTableBroker from IPublisherCore impl
  - Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl

* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
* Model config profiles after dotnet appsettings
  - Rename Configurations/<name>.json -> appsettings.<name>.json
  - Make json fields match corresponding csharp props verbatim
    * The Dependency Injection framework doesn't support JsonPropertyName
      attributes. Config class props must match json fields verbatim.

* Move reusable startup code into new Bts.Extensions.Hosting proj
* Add pubsub and md-publisher services to compose.yaml
This commit is contained in:
John Colvin committed 2026-01-04 21:08:11 +00:00
1 parent 716bc92394
commit 76fa35382e
102 files changed
+2948 -1169

No files matched your search

@@ -6,6 +6,8 @@ using System.Runtime.InteropServices;
using Bts.DataStreamExpressions;
using BCG.Utilities.General;
using bts;
using Bts.PubSub;
using Bts.DataStreamCore;
namespace BCG {
// listens to the requests posted on "cloud_streams" request table, and
@@ -42,13 +44,15 @@ namespace BCG {
private IDictionary<string, StreamPubUnit>
_stream_pub_map = new Dictionary<string, StreamPubUnit>();
private ViewStream _stream = new TableStream("cloud_streams");
private PubTableBroker _broker = new PubTableBroker();
private ManagedSubscriber _sub = ManagedSubscriber.instance;
readonly IPubTableBroker _broker;
readonly IManagedSubscriber _sub;
public RemoteQueryManager() {
public RemoteQueryManager(IManagedSubscriber managedSubscriber, IPubTableBroker pubTableBroker) {
_sub = managedSubscriber;
_broker = pubTableBroker;
_stream.onChange += _on_str;
_sub.subscribe("RemoteRequest", (string tag, byte[] data) => {
_sub.Subscribe("RemoteRequest", (string tag, byte[] data) => {
RemoteRequest req = RemoteRequest.construct_from_bytes(data);
_spawn_remote_stream(req.published_table_name, req.stream_expression, false);
});