John Colvin
0cf44fcc40
feat: Add historic market data streams factory
...
Loads historic md from Databento files in structured paths. It is
designed to work with files downloaded via databento_client.py
2025-11-15 12:45:04 -05:00
John Colvin
2549e34a17
Create LiveBBOStream via BookStreamFactory
...
Instead of PrimitiveManager
2025-11-15 12:42:40 -05:00
John Colvin
63665e9b10
refactor: Pass IMarketDataClient to streams via dependency injection
...
In general, the streams should not instantiate services, etc. themselves.
2025-11-15 12:42:40 -05:00
John Colvin
18b19ac787
refactor: Rewrite SortStream now that IBlueRowCollection is IList
2025-11-15 12:42:40 -05:00
John Colvin
805693e170
feat: Add ParquetReader/Writer streams
2025-11-15 12:40:48 -05:00
John Colvin
3a7f3ac608
fix: Fire multiple rows at once in replay bbo/trade streams doWork fn
...
If necessary to keep pace with timestamps in the file.
2025-11-15 12:38:51 -05:00
John Colvin
b82cc31d2d
refactor: Launch LiveTradeStream via IDataStreamFactory
2025-11-15 12:38:51 -05:00
John Colvin
144b29c728
improvement: Lookup timestamp col by index rather than name
2025-11-15 12:38:51 -05:00
John Colvin
e7625341e3
refactor: Remove hardcoded csv path from ReplayFutureLiveBBOStream
...
Add constructor overload to accept an input ViewStream
2025-11-15 12:38:51 -05:00
John Colvin
eb28e04bf1
refactor: Replace old NLStreamFactory w/ new BlueOrb one
...
Replace hardcoded file paths with proper config while we're at it
2025-11-15 12:38:50 -05:00
John Colvin
75f2678df6
cleanup: Fix build warnings
...
* BlueOrbNLStream.cs
* ChangeTableNameStream.cs
* FilterStream.cs
* FilterStream.cs
* GroupByOutputTable.cs
* GroupByStream.cs
* JoinForeignTable.cs
* MultiRowRollingMathStream.cs
* MutableAppendStream.cs
* PersistWindowState.cs
* PubsubEchoTestStream.cs
* RowDictionaryTable.cs
* SortedTable.cs
* TimeSeriesStream.cs
Also add Metadata prop to IDataPacket
2025-11-15 12:38:49 -05:00
John Colvin
6e1e5d4bbf
refactor: Add IDataStreamFactory(s) for spatial arb streams
2025-11-15 12:38:48 -05:00
John Colvin
cd291b7a0d
refactor: Hide md client behind interface, create via Dep Injection
2025-11-15 12:38:48 -05:00
John Colvin
c4f12be3e4
cleanup: Fix a ton of build warnings (cs)
...
All these warnings have been bothering me for a while. The work isn't done, but
this commit chops them down from ~1000 to ~600.
* AppendCollection.cs
* AppendStreamModified.cs
* BenzingaNewsStream.cs
* BenzingaNewsSymbolStateStream.cs
* BookStream.cs
* CSVSaverStream.cs
* ChartsUtils.cs
* ColumnInfo.cs
* ContractUtils.cs
* ContractUtils.cs
* DiffStream.cs
* FuturesActiveContractStream.cs
* FuzzyJoinOverTimeStream.cs
* HistoricalBarsStream.cs
* HistoricalBarsStream.cs
* InstrumentsStream.cs
* JoinOperandRowIndex.cs
* JoinRow.cs
* JoinStream.cs
* JoinTable.cs
* JoinTableSchema.cs
* LatencyStream.cs
* LeftInternalJoinStream.cs
* LightForeignJoinRow.cs
* LightForeignJoinRowList.cs
* LightLeftForeignJoinRow.cs
* LiteralStream.cs
* NLStream.cs
* NLToDSLStream.cs
* NLToIndexedFewShotPromptStream.cs
* NonIncGroupByStream.cs
* NonIncJoinBookkeeper.cs
* NonIncJoinSpec.cs
* OHLCVStream.cs
* OpenAIPromptStream.cs
* PriceStatsStream.cs
* PriceStatsStream.cs
* ProjectHelper.cs
* ProjectStream.cs
* PubSubServerStateStream.cs
* PubsubHeartbeatStream.cs
* SimpleNode.cs
* SortRow.cs
* SpatialArbStatsStream.cs
* SpatialFuturePairsStream.cs
* SpreadPerformanceEvaluator.cs
* SpreadPerformanceStream.cs
* StreamIOCommon.cs
* SuperJoinStream.cs
* TableMetaDataStream.cs
* TextWriterStream.cs
* TimeAverageStream.cs
2025-11-15 12:16:33 -05:00
John Colvin
54d7279832
style: Normalize indentation in PubsubHeartbeatStream.cs
2025-11-15 12:16:33 -05:00
John Colvin
0cc4b83417
cleanup: Remove redundant code block in HistoricalBarsStream.cs
2025-11-15 12:16:32 -05:00
John Colvin
e2816aaa33
refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore
2025-11-15 12:16:32 -05:00
John Colvin
e2a101eb20
feat: Add SpreadBook stream
2025-11-15 12:16:32 -05:00
John Colvin
05644ed6cf
feat: Add DataStream WebApp
...
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2025-11-14 15:00:45 -05:00
John Colvin
f119f30b62
fix load IDataStreamFactory(s) with depdendency injection
2025-11-14 14:57:43 -05:00
John Colvin
ada7d4bed1
refactor: Load IDataStreamFactory(s) with dependency injection
...
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-11-14 14:52:07 -05:00
John Colvin
41512cb811
fix build warnings: Make props nullable strings in SharedStreamConfig
2025-11-03 16:13:43 -05:00
John Colvin
f2cc576c51
Fix mismatched types in PubSubServerTagStatsStream schema
2025-11-03 16:13:18 -05:00
John Colvin
8db5655151
fix: Dispose market data sub in various md streams
2025-11-03 16:13:18 -05:00
John Colvin
96976b3fcd
cleanup: Modernize PubSubRoute code, json parsing
2025-11-03 16:13:17 -05:00
John Colvin
bdfef3d9b1
fix: Remove erroneous "DSL: " prefix in ai prompt stream
...
This causes parse errors since we extended the dsl syntax to support
named params in the form of "name: arg".
2025-11-03 16:13:16 -05:00
John Colvin
934ea76f5a
Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection
2025-11-03 16:13:16 -05:00
Taha Afzal
66c0baa150
Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln
2025-09-24 13:16:28 +00:00
arham
5305b32c04
Fix file path didnt work in linux
2025-09-17 04:28:25 +00:00
arham
e6304609f8
changes to make streams run on linux
2025-09-16 15:38:21 +00:00
arham
21d84d811d
simple changes to make stream work with wbesite
2025-09-02 17:17:43 +00:00
Taha
6b28fc4779
Lot of BlueOrb stream changes
2025-09-02 17:17:43 +00:00
Taha
0858eaac23
Polishing BlueOrb streams
2025-07-13 21:21:10 -04:00
Taha
70fae06a30
BlueOrb streams and MLP
2025-06-08 21:52:38 -04:00
Taha
a130e47218
Arb Scoring stream
2025-05-20 00:24:17 -04:00
Taha
0a6733180d
Added SpreadStatsStream
2025-05-18 01:10:33 -04:00
Taha
f2169c6bd4
Adding more data to SpreadPerformanceStream
2025-05-16 01:42:32 -04:00
Taha
23ef478391
Spread analytics streams
2025-05-15 15:01:27 -04:00
John Colvin
a6e8853cb9
fix build (warns): Account for nullable ref types
2025-04-28 17:12:58 +00:00
John Colvin
043efab5a2
build: Upgrade c# to net8.0
2025-04-28 17:12:58 +00:00
Taha
058b131760
Cleaning up some streams,new prompt etc
2025-04-04 15:36:52 -04:00
Taha
d13ba2968a
DateTimeExpanstionStream col name changes
2025-01-20 15:34:11 -05:00
Taha
461cbad338
Adding FakeBBO stream
2024-11-27 13:36:14 -05:00
Taha
aab8e83543
Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives
2024-11-26 22:33:26 -05:00
Taha
6163887cd3
Bunch of CME queries are now working
2024-10-30 13:36:23 -04:00
Taha
f9aa9dd982
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-10-15 23:48:34 -04:00
Taha
22f1ab44f9
Basic set of streams that can talks ot the LLM and get DSL back from few-shot prompt
2024-10-15 23:38:49 -04:00
Taha
288b7712db
Added and modified more streams to address teh CME NL queries
2024-10-09 13:59:01 -04:00
Taha
31df1a3827
Added benzinga streamd to MDPub's test config
2024-07-17 14:32:57 -04:00
Taha
7e219aec7c
Merging
2024-07-17 02:06:17 -04:00