Commit Graph
100 Commits
Author SHA1 Message Date
John Colvin ea0a1fbcd3 refactor: Load IDataStreamFactory(s) with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-11-19 13:53:18 -05:00
John Colvin 7566b2560f refactor: Change namespace to Bts.DataStreamCore for a few types
Since they're in the DataStreamCore csproj
2025-11-19 13:29:16 -05:00
John Colvin 7932ede1e8 refactor: Run RemoteQueryManager as an IHostedService
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2025-11-19 13:23:42 -05:00
John Colvin 12ea9a060a fix: Add a few null checks
* TimeBasedFilter
* TableAccumulator2
* NonIncGroupByStream
2025-11-19 13:21:51 -05:00
John Colvin 6e09854337 Add ErrorChanged event to propagate DataStream exceptions 2025-11-19 13:21:50 -05:00
John Colvin cbf1ef3d89 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-11-19 13:16:04 -05:00
John Colvin a86ddff4be refactor: Decouple ViewPublisher from IPubTableBroker impl 2025-11-19 12:27:23 -05:00
John Colvin f51d0ae973 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2025-11-19 12:27:20 -05:00
John Colvin b9125fb44b refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2025-11-19 12:24:39 -05:00
John Colvin b2e94db451 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2025-11-19 12:24:38 -05:00
John Colvin 77c616af3e cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-11-19 12:24:38 -05:00
John Colvin 4d0d7cf279 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-11-19 12:24:37 -05:00
John Colvin f571b4992f Add generic ViewStream, BlueRow types/interfaces 2025-11-19 12:24:37 -05:00
John Colvin ca82b8f001 fix: Remove query from literal error stream
The parser can't handle colons in the expression (it throws/crashes)
2025-11-19 12:24:36 -05:00
John Colvin 706d1b8de4 Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection 2025-11-19 12:24:36 -05:00
John Colvin 5080960b97 refactor: Replace DataPacket w/ IDataPacket in stream handlers
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-19 00:36:56 -05:00
Taha Afzal 09abe4a432 Added RowBasedAppendStream 2025-11-18 03:10:30 +00:00
arham 5305b32c04 Fix file path didnt work in linux 2025-09-17 04:28:25 +00:00
arham e6304609f8 changes to make streams run on linux 2025-09-16 15:38:21 +00:00
Taha 6b28fc4779 Lot of BlueOrb stream changes 2025-09-02 17:17:43 +00:00
Taha afd229b7c7 Merging 2025-07-15 17:51:28 -04:00
John Colvin 7e79a21cd4 Add some nifty DataPacket/TableUpdate IObservable utils 2025-07-15 10:13:53 -04:00
John Colvin 77caef2640 fix build: Add missing name to fn arg 2025-07-15 10:13:53 -04:00
Taha 767fa25d61 Adding comments 2025-07-14 15:18:03 -04:00
Taha 0858eaac23 Polishing BlueOrb streams 2025-07-13 21:21:10 -04:00
Taha 70fae06a30 BlueOrb streams and MLP 2025-06-08 21:52:38 -04:00
Taha a130e47218 Arb Scoring stream 2025-05-20 00:24:17 -04:00
Taha 0a6733180d Added SpreadStatsStream 2025-05-18 01:10:33 -04:00
Taha 23ef478391 Spread analytics streams 2025-05-15 15:01:27 -04:00
John Colvin 043efab5a2 build: Upgrade c# to net8.0 2025-04-28 17:12:58 +00:00
Taha 058b131760 Cleaning up some streams,new prompt etc 2025-04-04 15:36:52 -04:00
Taha d13ba2968a DateTimeExpanstionStream col name changes 2025-01-20 15:34:11 -05:00
Taha 461cbad338 Adding FakeBBO stream 2024-11-27 13:36:14 -05:00
Taha aab8e83543 Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives 2024-11-26 22:33:26 -05:00
Taha 6163887cd3 Bunch of CME queries are now working 2024-10-30 13:36:23 -04:00
Taha f9aa9dd982 Merge branch 'development' of bitbucket.org:bluetradesys/spark into development 2024-10-15 23:48:34 -04:00
Taha 22f1ab44f9 Basic set of streams that can talks ot the LLM and get DSL back from few-shot prompt 2024-10-15 23:38:49 -04:00
Taha 288b7712db Added and modified more streams to address teh CME NL queries 2024-10-09 13:59:01 -04:00
Taha 322c06bfbe Fixing build relted to BCG vs bts namespace 2024-08-21 16:24:22 -04:00
Taha b68fceb362 Cleaning up FileStreams 2024-08-21 16:11:57 -04:00
Taha f06beff22f Added new benzinga streams 2024-07-17 02:02:01 -04:00
Taha 050e21e22b Adding TableMetaDataStream 2024-07-04 23:39:04 -04:00
Taha Afzal 672ad6d237 Cleaning Config 1.0 (not completely done) 2024-06-30 23:17:20 -04:00
Taha 393f304be4 some buf gixes + cleanup 2024-06-26 00:23:19 -04:00
Taha 87b6a2044c PubSubServer's linux issues fixed 2024-06-23 21:13:37 -04:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d6fae238f9 Add IObservable ViewStream wrapper 2024-06-03 13:46:51 +00:00
John Colvin 5741ec9ab2 cleanup: Remove unnecessary usings (namespaces) 2024-06-03 13:46:51 +00:00