John Colvin
d2a1b67717
refactor: Make IBlueRowCollection inherit IList<IBlueRow>
2025-11-15 12:38:51 -05:00
John Colvin
fb15b3fb24
improvement: Don't mutate IBlueRowCollection after firing in FilterStream
2025-11-15 12:38:51 -05:00
John Colvin
ea0246929a
improvement: Make MultiRowRollingMathStream more performant
...
Avoid creating extra lists and o(n) enumeration
2025-11-15 12:38:50 -05:00
John Colvin
52f35cc421
refactor: Move IBlueTableSchema ext fns into DataStreamCore
...
The csproj in which IBlueTableSchema is defined.
2025-11-15 12:38:50 -05:00
John Colvin
0d10c2ba5d
new: Add GetColumn fn to IBlueRowCollection interface
...
This allows us to implement python/pandas functionality, much of which is based
on column operations.
2025-11-15 12:38:50 -05:00
John Colvin
75f2678df6
cleanup: Fix build warnings
...
* BlueOrbNLStream.cs
* ChangeTableNameStream.cs
* FilterStream.cs
* FilterStream.cs
* GroupByOutputTable.cs
* GroupByStream.cs
* JoinForeignTable.cs
* MultiRowRollingMathStream.cs
* MutableAppendStream.cs
* PersistWindowState.cs
* PubsubEchoTestStream.cs
* RowDictionaryTable.cs
* SortedTable.cs
* TimeSeriesStream.cs
Also add Metadata prop to IDataPacket
2025-11-15 12:38:49 -05:00
John Colvin
f07d83e7bb
refactor: Make CompositeDataStreamFactory class internal
2025-11-15 12:38:49 -05:00
John Colvin
760cdc0fba
cleanup: Fix build warnings, remove unused/redundant code
...
* Make a few member vars and fns local to the constructor
* Replace a couple non-generic collections with generic ones
* Fix warnings and clean code in:
- DictionaryExtensions.cs
* Remove ext fns that are part of dotnet now
- JoinNameResolver.cs
- MainForm.cs
cleanup: Remove unused code in ladder
Some props in cs, and StaticResources in xaml were leftover after I combined
the panel price points into a single collection, bound to an ItemsControl.
2025-11-15 12:38:49 -05:00
John Colvin
29080795f7
refactor: Rename AddRemoteQueryManager -> AddRemoteQueryStreams
2025-11-15 12:38:48 -05:00
John Colvin
c4f12be3e4
cleanup: Fix a ton of build warnings (cs)
...
All these warnings have been bothering me for a while. The work isn't done, but
this commit chops them down from ~1000 to ~600.
* AppendCollection.cs
* AppendStreamModified.cs
* BenzingaNewsStream.cs
* BenzingaNewsSymbolStateStream.cs
* BookStream.cs
* CSVSaverStream.cs
* ChartsUtils.cs
* ColumnInfo.cs
* ContractUtils.cs
* ContractUtils.cs
* DiffStream.cs
* FuturesActiveContractStream.cs
* FuzzyJoinOverTimeStream.cs
* HistoricalBarsStream.cs
* HistoricalBarsStream.cs
* InstrumentsStream.cs
* JoinOperandRowIndex.cs
* JoinRow.cs
* JoinStream.cs
* JoinTable.cs
* JoinTableSchema.cs
* LatencyStream.cs
* LeftInternalJoinStream.cs
* LightForeignJoinRow.cs
* LightForeignJoinRowList.cs
* LightLeftForeignJoinRow.cs
* LiteralStream.cs
* NLStream.cs
* NLToDSLStream.cs
* NLToIndexedFewShotPromptStream.cs
* NonIncGroupByStream.cs
* NonIncJoinBookkeeper.cs
* NonIncJoinSpec.cs
* OHLCVStream.cs
* OpenAIPromptStream.cs
* PriceStatsStream.cs
* PriceStatsStream.cs
* ProjectHelper.cs
* ProjectStream.cs
* PubSubServerStateStream.cs
* PubsubHeartbeatStream.cs
* SimpleNode.cs
* SortRow.cs
* SpatialArbStatsStream.cs
* SpatialFuturePairsStream.cs
* SpreadPerformanceEvaluator.cs
* SpreadPerformanceStream.cs
* StreamIOCommon.cs
* SuperJoinStream.cs
* TableMetaDataStream.cs
* TextWriterStream.cs
* TimeAverageStream.cs
2025-11-15 12:16:33 -05:00
John Colvin
4f1efea066
cleanup: Fix indentation and build warnings in TestStream.cs
2025-11-15 12:16:32 -05:00
John Colvin
e2816aaa33
refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore
2025-11-15 12:16:32 -05:00
John Colvin
45fcb2e20b
refactor: Use HostBuilder to launch spark w/ streams
2025-11-15 12:16:32 -05:00
John Colvin
15883e42b6
fix (dsqa): Resolve name conflicts with new BlueOrb streams
2025-11-14 15:02:16 -05:00
John Colvin
34caf54015
fix: Fix typo in "DateTimeExpansionStream" type name
...
Previously, it was "DateTimeExpanstion" (erroneous "t")
2025-11-14 15:02:16 -05:00
John Colvin
d6a88a7ac9
cleanup: Remove unused code
2025-11-14 14:59:33 -05:00
John Colvin
519ef8b0af
Increase SQLiteEnumerator page size to 65536
...
This performs better, and doesn't seem to be too large in practice
2025-11-14 14:59:33 -05:00
John Colvin
ada7d4bed1
refactor: Load IDataStreamFactory(s) with dependency injection
...
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-11-14 14:52:07 -05:00
John Colvin
f65b6b82f2
refactor: Replace DataPacket w/ IDataPacket in stream handlers
...
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-03 16:21:32 -05:00
John Colvin
1155fd6997
refactor: Change namespace to Bts.DataStreamCore for a few types
...
Since they're in the DataStreamCore csproj
2025-11-03 16:19:50 -05:00
John Colvin
1481a3fd07
refactor: Run RemoteQueryManager as an IHostedService
...
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2025-11-03 16:19:45 -05:00
John Colvin
d669d24aea
fix: Add a few null checks
...
* TimeBasedFilter
* TableAccumulator2
* NonIncGroupByStream
2025-11-03 16:13:43 -05:00
John Colvin
cd7c04dce5
Add ErrorChanged event to propagate DataStream exceptions
2025-11-03 16:13:19 -05:00
John Colvin
b25038fac6
Add DataStreamPublisher service modules designed for Dependency Injection
...
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-11-03 16:13:18 -05:00
John Colvin
9b95caa8e6
refactor: Decouple ViewPublisher from IPubTableBroker impl
2025-11-03 16:13:18 -05:00
John Colvin
b4cb6ac856
Add PubSub service modules designed for Dependency Injection
...
* Get rid of JsonPropertyName attributes in PubSubConfig
- The Dependency Injection framework (based around dotnet ApplicationBuilder)
does not support this attribute, and maps property names verbatim to json fields.
* Change PubSubServer json fields to match cs class property names verbatim
2025-11-03 16:13:18 -05:00
John Colvin
83cbb0f47f
refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl
2025-11-03 16:13:18 -05:00
John Colvin
f240f005f4
refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
...
This matches the file name
2025-11-03 16:13:18 -05:00
John Colvin
6e77eda16c
cleanup: Fix numerous build warnings in DataStreamSuite
...
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.
* PubSubClientMessage
- Normalize indentation/whitespace
- Fix some typos
- Address nullability issues to fix build warnings
- Modernize code
* Compression related files
- ZipEntry
- ZipFile
- ZipInputStream
- TarHeader
* RemoteMessage
- make 'message' nullable to resolve warnings elsewhere (in calling code)
* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-11-03 16:13:17 -05:00
John Colvin
41be31a7b0
cleanup: Disable nullability warnings
...
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-11-03 16:13:17 -05:00
John Colvin
bf1ce9467d
Add generic ViewStream, BlueRow types/interfaces
2025-11-03 16:13:16 -05:00
John Colvin
33764aaa28
fix: Remove query from literal error stream
...
The parser can't handle colons in the expression (it throws/crashes)
2025-11-03 16:13:16 -05:00
John Colvin
934ea76f5a
Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection
2025-11-03 16:13:16 -05:00
arham
5305b32c04
Fix file path didnt work in linux
2025-09-17 04:28:25 +00:00
arham
e6304609f8
changes to make streams run on linux
2025-09-16 15:38:21 +00:00
Taha
6b28fc4779
Lot of BlueOrb stream changes
2025-09-02 17:17:43 +00:00
Taha
afd229b7c7
Merging
2025-07-15 17:51:28 -04:00
John Colvin
7e79a21cd4
Add some nifty DataPacket/TableUpdate IObservable utils
2025-07-15 10:13:53 -04:00
John Colvin
77caef2640
fix build: Add missing name to fn arg
2025-07-15 10:13:53 -04:00
Taha
767fa25d61
Adding comments
2025-07-14 15:18:03 -04:00
Taha
0858eaac23
Polishing BlueOrb streams
2025-07-13 21:21:10 -04:00
Taha
70fae06a30
BlueOrb streams and MLP
2025-06-08 21:52:38 -04:00
Taha
a130e47218
Arb Scoring stream
2025-05-20 00:24:17 -04:00
Taha
0a6733180d
Added SpreadStatsStream
2025-05-18 01:10:33 -04:00
Taha
23ef478391
Spread analytics streams
2025-05-15 15:01:27 -04:00
John Colvin
043efab5a2
build: Upgrade c# to net8.0
2025-04-28 17:12:58 +00:00
Taha
058b131760
Cleaning up some streams,new prompt etc
2025-04-04 15:36:52 -04:00
Taha
d13ba2968a
DateTimeExpanstionStream col name changes
2025-01-20 15:34:11 -05:00
Taha
461cbad338
Adding FakeBBO stream
2024-11-27 13:36:14 -05:00
Taha
aab8e83543
Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives
2024-11-26 22:33:26 -05:00