Commit Graph
116 Commits
Author SHA1 Message Date
John Colvin d2a1b67717 refactor: Make IBlueRowCollection inherit IList<IBlueRow> 2025-11-15 12:38:51 -05:00
John Colvin fb15b3fb24 improvement: Don't mutate IBlueRowCollection after firing in FilterStream 2025-11-15 12:38:51 -05:00
John Colvin ea0246929a improvement: Make MultiRowRollingMathStream more performant
Avoid creating extra lists and o(n) enumeration
2025-11-15 12:38:50 -05:00
John Colvin 52f35cc421 refactor: Move IBlueTableSchema ext fns into DataStreamCore
The csproj in which IBlueTableSchema is defined.
2025-11-15 12:38:50 -05:00
John Colvin 0d10c2ba5d new: Add GetColumn fn to IBlueRowCollection interface
This allows us to implement python/pandas functionality, much of which is based
on column operations.
2025-11-15 12:38:50 -05:00
John Colvin 75f2678df6 cleanup: Fix build warnings
* BlueOrbNLStream.cs
* ChangeTableNameStream.cs
* FilterStream.cs
* FilterStream.cs
* GroupByOutputTable.cs
* GroupByStream.cs
* JoinForeignTable.cs
* MultiRowRollingMathStream.cs
* MutableAppendStream.cs
* PersistWindowState.cs
* PubsubEchoTestStream.cs
* RowDictionaryTable.cs
* SortedTable.cs
* TimeSeriesStream.cs

Also add Metadata prop to IDataPacket
2025-11-15 12:38:49 -05:00
John Colvin f07d83e7bb refactor: Make CompositeDataStreamFactory class internal 2025-11-15 12:38:49 -05:00
John Colvin 760cdc0fba cleanup: Fix build warnings, remove unused/redundant code
* Make a few member vars and fns local to the constructor
* Replace a couple non-generic collections with generic ones
* Fix warnings and clean code in:
  - DictionaryExtensions.cs
    * Remove ext fns that are part of dotnet now
  - JoinNameResolver.cs
  - MainForm.cs

cleanup: Remove unused code in ladder

Some props in cs, and StaticResources in xaml were leftover after I combined
the panel price points into a single collection, bound to an ItemsControl.
2025-11-15 12:38:49 -05:00
John Colvin 29080795f7 refactor: Rename AddRemoteQueryManager -> AddRemoteQueryStreams 2025-11-15 12:38:48 -05:00
John Colvin c4f12be3e4 cleanup: Fix a ton of build warnings (cs)
All these warnings have been bothering me for a while. The work isn't done, but
this commit chops them down from ~1000 to ~600.

* AppendCollection.cs
* AppendStreamModified.cs
* BenzingaNewsStream.cs
* BenzingaNewsSymbolStateStream.cs
* BookStream.cs
* CSVSaverStream.cs
* ChartsUtils.cs
* ColumnInfo.cs
* ContractUtils.cs
* ContractUtils.cs
* DiffStream.cs
* FuturesActiveContractStream.cs
* FuzzyJoinOverTimeStream.cs
* HistoricalBarsStream.cs
* HistoricalBarsStream.cs
* InstrumentsStream.cs
* JoinOperandRowIndex.cs
* JoinRow.cs
* JoinStream.cs
* JoinTable.cs
* JoinTableSchema.cs
* LatencyStream.cs
* LeftInternalJoinStream.cs
* LightForeignJoinRow.cs
* LightForeignJoinRowList.cs
* LightLeftForeignJoinRow.cs
* LiteralStream.cs
* NLStream.cs
* NLToDSLStream.cs
* NLToIndexedFewShotPromptStream.cs
* NonIncGroupByStream.cs
* NonIncJoinBookkeeper.cs
* NonIncJoinSpec.cs
* OHLCVStream.cs
* OpenAIPromptStream.cs
* PriceStatsStream.cs
* PriceStatsStream.cs
* ProjectHelper.cs
* ProjectStream.cs
* PubSubServerStateStream.cs
* PubsubHeartbeatStream.cs
* SimpleNode.cs
* SortRow.cs
* SpatialArbStatsStream.cs
* SpatialFuturePairsStream.cs
* SpreadPerformanceEvaluator.cs
* SpreadPerformanceStream.cs
* StreamIOCommon.cs
* SuperJoinStream.cs
* TableMetaDataStream.cs
* TextWriterStream.cs
* TimeAverageStream.cs
2025-11-15 12:16:33 -05:00
John Colvin 4f1efea066 cleanup: Fix indentation and build warnings in TestStream.cs 2025-11-15 12:16:32 -05:00
John Colvin e2816aaa33 refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore 2025-11-15 12:16:32 -05:00
John Colvin 45fcb2e20b refactor: Use HostBuilder to launch spark w/ streams 2025-11-15 12:16:32 -05:00
John Colvin 15883e42b6 fix (dsqa): Resolve name conflicts with new BlueOrb streams 2025-11-14 15:02:16 -05:00
John Colvin 34caf54015 fix: Fix typo in "DateTimeExpansionStream" type name
Previously, it was "DateTimeExpanstion" (erroneous "t")
2025-11-14 15:02:16 -05:00
John Colvin d6a88a7ac9 cleanup: Remove unused code 2025-11-14 14:59:33 -05:00
John Colvin 519ef8b0af Increase SQLiteEnumerator page size to 65536
This performs better, and doesn't seem to be too large in practice
2025-11-14 14:59:33 -05:00
John Colvin ada7d4bed1 refactor: Load IDataStreamFactory(s) with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-11-14 14:52:07 -05:00
John Colvin f65b6b82f2 refactor: Replace DataPacket w/ IDataPacket in stream handlers
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-03 16:21:32 -05:00
John Colvin 1155fd6997 refactor: Change namespace to Bts.DataStreamCore for a few types
Since they're in the DataStreamCore csproj
2025-11-03 16:19:50 -05:00
John Colvin 1481a3fd07 refactor: Run RemoteQueryManager as an IHostedService
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2025-11-03 16:19:45 -05:00
John Colvin d669d24aea fix: Add a few null checks
* TimeBasedFilter
* TableAccumulator2
* NonIncGroupByStream
2025-11-03 16:13:43 -05:00
John Colvin cd7c04dce5 Add ErrorChanged event to propagate DataStream exceptions 2025-11-03 16:13:19 -05:00
John Colvin b25038fac6 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-11-03 16:13:18 -05:00
John Colvin 9b95caa8e6 refactor: Decouple ViewPublisher from IPubTableBroker impl 2025-11-03 16:13:18 -05:00
John Colvin b4cb6ac856 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2025-11-03 16:13:18 -05:00
John Colvin 83cbb0f47f refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2025-11-03 16:13:18 -05:00
John Colvin f240f005f4 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2025-11-03 16:13:18 -05:00
John Colvin 6e77eda16c cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-11-03 16:13:17 -05:00
John Colvin 41be31a7b0 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-11-03 16:13:17 -05:00
John Colvin bf1ce9467d Add generic ViewStream, BlueRow types/interfaces 2025-11-03 16:13:16 -05:00
John Colvin 33764aaa28 fix: Remove query from literal error stream
The parser can't handle colons in the expression (it throws/crashes)
2025-11-03 16:13:16 -05:00
John Colvin 934ea76f5a Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection 2025-11-03 16:13:16 -05:00
arham 5305b32c04 Fix file path didnt work in linux 2025-09-17 04:28:25 +00:00
arham e6304609f8 changes to make streams run on linux 2025-09-16 15:38:21 +00:00
Taha 6b28fc4779 Lot of BlueOrb stream changes 2025-09-02 17:17:43 +00:00
Taha afd229b7c7 Merging 2025-07-15 17:51:28 -04:00
John Colvin 7e79a21cd4 Add some nifty DataPacket/TableUpdate IObservable utils 2025-07-15 10:13:53 -04:00
John Colvin 77caef2640 fix build: Add missing name to fn arg 2025-07-15 10:13:53 -04:00
Taha 767fa25d61 Adding comments 2025-07-14 15:18:03 -04:00
Taha 0858eaac23 Polishing BlueOrb streams 2025-07-13 21:21:10 -04:00
Taha 70fae06a30 BlueOrb streams and MLP 2025-06-08 21:52:38 -04:00
Taha a130e47218 Arb Scoring stream 2025-05-20 00:24:17 -04:00
Taha 0a6733180d Added SpreadStatsStream 2025-05-18 01:10:33 -04:00
Taha 23ef478391 Spread analytics streams 2025-05-15 15:01:27 -04:00
John Colvin 043efab5a2 build: Upgrade c# to net8.0 2025-04-28 17:12:58 +00:00
Taha 058b131760 Cleaning up some streams,new prompt etc 2025-04-04 15:36:52 -04:00
Taha d13ba2968a DateTimeExpanstionStream col name changes 2025-01-20 15:34:11 -05:00
Taha 461cbad338 Adding FakeBBO stream 2024-11-27 13:36:14 -05:00
Taha aab8e83543 Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives 2024-11-26 22:33:26 -05:00