Commit Graph
91 Commits
Author SHA1 Message Date
John Colvin bb1e7c30c8 build: Add msbuild flag to reference projs or local nugets
So we can use project refs for dev builds, and package refs for releases.
2026-01-04 21:19:00 +00:00
John Colvin f60782915c refactor: Rename spark_core csproj Bts.Spark.Core
This follows the conventional naming practices, where the project shares its
name with the namespace. Doing this in anticipation of the csharp sdk we are
about to release (for Ophir).
2026-01-04 21:08:13 +00:00
John Colvin 7faa0ade24 diag: Assert that local ip address is successfully obtained 2026-01-04 21:08:12 +00:00
John Colvin 4d6d68aff6 improve: Resolve hostnames when comparing ip addrs
So we can to configure pubsub routes by hostname or ipv4 (ipv6 hasn't
been tested yet...so we should assume it doesn't work).
2026-01-04 21:08:12 +00:00
John Colvin dcf291d6b2 diag: Assert TcpConnection endpoint matches expected 2026-01-04 21:08:12 +00:00
John Colvin a6b83988e7 diag: Catch exceptions on AsyncTCPConnector_AsyncWrapper bg thread
Since it spawns a new thread on each attempt, uncaught exceptions were
previously undetected, and caused silent failures down the pipe. This
should make it easier to track such bugs down.
2026-01-04 21:08:12 +00:00
John Colvin 2c097500f4 fix: Format PubSubRoute as json in log msgs
It was previously using the default ToString, which simply returns the type name
2026-01-04 21:08:12 +00:00
John Colvin 0253a0f60f refactor: Run RemoteQueryManager as an IHostedService
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2026-01-04 21:08:12 +00:00
John Colvin a7b6da27ae fix: Catch exception when creating TcpClient 2026-01-04 21:08:12 +00:00
John Colvin de4a55b34c fix: Make SystemSubscriptionMapPublisher threadsafe
The PubSubState.onTagMapChanged event may not be invoked on
the same thread as doWork.
2026-01-04 21:08:12 +00:00
John Colvin 3038d9154f improve: Reduce task wait/timeout code in AsyncTCPConnector 2026-01-04 21:08:12 +00:00
John Colvin 5805758187 diag: Assert System publishers 'doWork' is always called on same thread 2026-01-04 21:08:12 +00:00
John Colvin 60af47c4eb cleanup: Remove unused PubSubConfig props 2026-01-04 21:08:12 +00:00
John Colvin b0d0594e3d fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2026-01-04 21:08:12 +00:00
John Colvin 76fa35382e refactor: Add DataStream service modules for Dependency Injection
Namely, in DataStreamPublisher and PubSubServer. This is part of a broader
paradigm shift towards Dependency Injection and Factory Pattern.

https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection-guidelines

* Use interfaces to decouple services:
  - Decouple ManagedSubscriber from ISubscriberCore impl
  - Decouple PubTableBroker from IPublisherCore impl
  - Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl

* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
* Model config profiles after dotnet appsettings
  - Rename Configurations/<name>.json -> appsettings.<name>.json
  - Make json fields match corresponding csharp props verbatim
    * The Dependency Injection framework doesn't support JsonPropertyName
      attributes. Config class props must match json fields verbatim.

* Move reusable startup code into new Bts.Extensions.Hosting proj
* Add pubsub and md-publisher services to compose.yaml
2026-01-04 21:08:11 +00:00
John Colvin 716bc92394 refactor: Only support 1 tag type in IPubSubServerTagHandler 2025-12-30 11:42:23 -05:00
John Colvin 7e32ac9e06 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2025-12-30 11:38:21 -05:00
John Colvin cc483ca4b5 diag: Fix typos in log msgs
* Bad string interopolation syntax
2025-12-30 11:38:20 -05:00
John Colvin 7d6e4efa4f cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* Compression related files
  - BZip2InputStream  (disable nullability warnings)
  - BZip2OutputStream (disable nullability warnings)
  - DeflaterHuffman   (disable nullability warnings)
  - Inflater          (disable nullability warnings)
  - InflaterDynHeader (disable nullability warnings)
  - ParallelJoinTest  (disable nullability warnings)
  - TarHeader
  - TarInputStream    (disable nullability warnings)
  - ZipEntry
  - ZipFile
  - ZipInputStream

* AsyncTCPConnection_APM
* AsyncTCPConnection_APM
* InflaterHuffmanTree
* ManagedSubscriber
* ManagedTCPMessageHub
* MapStream
* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code
* PubSubMessageReactor
* PubSubServerManagedTCPConnection
* PubSubSingleTagStateEvent
* PubSubState
* PublisherCore
* PubsubDeserializationController
* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)
* SharedStreamConfig
* SingleTagData
* SubscriberCore
* SuperMapStream
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SystemSubscriptionMapPublisher
* ThreadSafeQueue
* TradeBarsStream

Only use Registry on windows.
2025-12-30 11:38:20 -05:00
John Colvin 02f566a970 cleanup: Modernize PubSubRoute code, json parsing 2025-12-30 11:38:20 -05:00
John Colvin 2014931653 feat: Add live bbo stream 2025-11-19 10:33:31 -05:00
John Colvin 043efab5a2 build: Upgrade c# to net8.0 2025-04-28 17:12:58 +00:00
John Colvin 37e5003cf9 Merge remote-tracking branch 'origin/development.ps_linux' into development 2024-09-19 05:40:59 +00:00
Taha b68fceb362 Cleaning up FileStreams 2024-08-21 16:11:57 -04:00
Taha 7e219aec7c Merging 2024-07-17 02:06:17 -04:00
Taha f06beff22f Added new benzinga streams 2024-07-17 02:02:01 -04:00
Taha Afzal 6aa39a716a PubSubSingleTagState is not needed any more 2024-07-16 01:15:51 +00:00
Taha Afzal 5088b5ebe0 Fixing problems where PSS's client socket's IP was different than what the client was sending in PubSubClientSubscription 2024-07-08 23:09:53 +00:00
Taha Afzal 233e2576af Merging 2024-07-05 21:21:27 +00:00
Taha Afzal 543c655bca src/DataStreamSuite/PubSub/ManagedTCP/AsyncTCPConnection_AsyncWrapper.cs 2024-07-05 21:13:36 +00:00
Taha Afzal 672ad6d237 Cleaning Config 1.0 (not completely done) 2024-06-30 23:17:20 -04:00
Taha b177ffef04 Fixing the bug where messages were being sent by the pubsubserver out of order 2024-06-27 00:17:22 -04:00
Taha 393f304be4 some buf gixes + cleanup 2024-06-26 00:23:19 -04:00
Taha 151816bf5e PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop 2024-06-24 02:29:28 -04:00
Taha 5cf7eacd82 Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe 2024-06-24 00:28:07 -04:00
Taha 87b6a2044c PubSubServer's linux issues fixed 2024-06-23 21:13:37 -04:00
John Colvin 3b631e0d5f Fix a couple more typos in logs 2024-06-07 16:35:06 +00:00
John Colvin 23d7d3775d cleanup: Delete unused files 2024-06-07 10:40:34 -05:00
John Colvin 8c797ee76b Fix typo in log msg 2024-06-06 09:21:01 -05:00
John Colvin 068c1c6973 cleanup: Remove erroneous comment 2024-06-06 09:21:01 -05:00
John Colvin 4a2735839e Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
Taha 07bade53c0 Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB 2024-06-06 09:20:05 -05:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin a3ca06addc Reduce 'internal_publish' log level to Diag
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin 32869e54ec fix: Fix typos in c# Logs
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin 17e60df32b cleanup: Remove unused file PubSub3Config.cs 2024-06-06 09:15:16 -05:00
John Colvin c99d5fe24c Add support for core PubSub config via json 2024-06-06 09:15:16 -05:00
John Colvin 8a3e4edb4d Dispose connections in SimpleTCPServer.Dispose 2024-06-06 09:15:15 -05:00
John Colvin 4855935ae0 improvement: Get PubSubRoutes from config (json)
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin 549708fcc2 fix: Use ".dll" file ext when loading assemblies by path 2024-06-05 15:44:35 +00:00