John Colvin
03294ffe37
feat: Add function defs to DataStream DSL
2025-11-15 12:42:39 -05:00
John Colvin
0550078de0
chore: Support mutations in AggregateDictionary (optionally)
2025-11-15 12:42:39 -05:00
John Colvin
89129f726e
chore: Implement IComparable in BookTuple
2025-11-15 12:42:39 -05:00
John Colvin
97eefb1a1e
improve: Serialize to/from parquet in PythonDataframeStream
2025-11-15 12:42:39 -05:00
John Colvin
805693e170
feat: Add ParquetReader/Writer streams
2025-11-15 12:40:48 -05:00
John Colvin
3a7f3ac608
fix: Fire multiple rows at once in replay bbo/trade streams doWork fn
...
If necessary to keep pace with timestamps in the file.
2025-11-15 12:38:51 -05:00
John Colvin
b82cc31d2d
refactor: Launch LiveTradeStream via IDataStreamFactory
2025-11-15 12:38:51 -05:00
John Colvin
34859d7506
chore: Add id field to TradeRecord
...
Need this for streams
2025-11-15 12:38:51 -05:00
John Colvin
8a9bc1872e
fix: Actually use sec filter in MarketDataClient.Trades
2025-11-15 12:38:51 -05:00
John Colvin
6b4ff90a56
style: Rename dispose_thread_safe in PascalCase
2025-11-15 12:38:51 -05:00
John Colvin
d2a1b67717
refactor: Make IBlueRowCollection inherit IList<IBlueRow>
2025-11-15 12:38:51 -05:00
John Colvin
144b29c728
improvement: Lookup timestamp col by index rather than name
2025-11-15 12:38:51 -05:00
John Colvin
e7625341e3
refactor: Remove hardcoded csv path from ReplayFutureLiveBBOStream
...
Add constructor overload to accept an input ViewStream
2025-11-15 12:38:51 -05:00
John Colvin
fb15b3fb24
improvement: Don't mutate IBlueRowCollection after firing in FilterStream
2025-11-15 12:38:51 -05:00
John Colvin
ea0246929a
improvement: Make MultiRowRollingMathStream more performant
...
Avoid creating extra lists and o(n) enumeration
2025-11-15 12:38:50 -05:00
John Colvin
52f35cc421
refactor: Move IBlueTableSchema ext fns into DataStreamCore
...
The csproj in which IBlueTableSchema is defined.
2025-11-15 12:38:50 -05:00
John Colvin
7e968fec76
tool: Add databento_client.py to fetch (and store) databento data
2025-11-15 12:38:50 -05:00
John Colvin
188acc08ca
chore: Copy/paste some log utils from py interop dir to tools/py
2025-11-15 12:38:50 -05:00
John Colvin
29be8c3ff5
refactor: Support dict and list of tuples in py fmt_table
2025-11-15 12:38:50 -05:00
John Colvin
eb28e04bf1
refactor: Replace old NLStreamFactory w/ new BlueOrb one
...
Replace hardcoded file paths with proper config while we're at it
2025-11-15 12:38:50 -05:00
John Colvin
4ef2dbe4d7
refactor: Pass composite stream factory back to recursive factories as service
2025-11-15 12:38:50 -05:00
John Colvin
979051f147
build: Add appsettings.Local.json to DSQA
2025-11-15 12:38:50 -05:00
John Colvin
35a17b0fd0
refactor: Support random access (IList) to data in TableUpdate
...
We're doing the same thing to IBlueRowCollection. This will allow us to
implement column based operators, like python pandas.
2025-11-15 12:38:50 -05:00
John Colvin
d93ce336f6
Add TestCommand back to Spark.Console cli
2025-11-15 12:38:50 -05:00
John Colvin
0d10c2ba5d
new: Add GetColumn fn to IBlueRowCollection interface
...
This allows us to implement python/pandas functionality, much of which is based
on column operations.
2025-11-15 12:38:50 -05:00
John Colvin
2a4d62a1e9
chore: Add various collection classes
...
These will be useful in stream operators, row indexing, etc.
* AggregateList
* DeltaList
* DictionarySetWrapper
* ImmutableOrderedDictionary
* IMutableListFacade
- inherits IReadOnlyList<T> and IList<T>
- throws NotSupportedException in default impls for mutation ops
* IMutableCollectionFacade
- inherits IReadOnlyCollection<T> and ICollection<T>
- throws NotSupportedException in default impls for mutation ops
* MergedSortedList
* OrderedDictionary
* ReverseArrayList
* SliceList
* StepList
2025-11-15 12:38:50 -05:00
John Colvin
b9e191eeb9
refactor: Drop nullable modifier for TValue in GetValueOrDefault ext fn
2025-11-15 12:38:50 -05:00
John Colvin
75f2678df6
cleanup: Fix build warnings
...
* BlueOrbNLStream.cs
* ChangeTableNameStream.cs
* FilterStream.cs
* FilterStream.cs
* GroupByOutputTable.cs
* GroupByStream.cs
* JoinForeignTable.cs
* MultiRowRollingMathStream.cs
* MutableAppendStream.cs
* PersistWindowState.cs
* PubsubEchoTestStream.cs
* RowDictionaryTable.cs
* SortedTable.cs
* TimeSeriesStream.cs
Also add Metadata prop to IDataPacket
2025-11-15 12:38:49 -05:00
John Colvin
8bbe850549
chore: Add 'Rows' prop to BlueTable to avoid accidentally iterating over KeyValuePairs
2025-11-15 12:38:49 -05:00
John Colvin
25860af7bf
refactor: Use dependency injection in Spark.Console
2025-11-15 12:38:49 -05:00
John Colvin
68198a058e
cleanup: Remove broken settings cmd from Spark.Console
2025-11-15 12:38:49 -05:00
John Colvin
7297d667a2
refactor: Use top-level statements for Spark.Console 'main'
2025-11-15 12:38:49 -05:00
John Colvin
ace1b5f04d
refactor: Consolidate SparkApp and ShellBootstrapper into single class
2025-11-15 12:38:49 -05:00
John Colvin
bd252c251c
feat: Add stream driven price lines to ladder
2025-11-15 12:38:49 -05:00
John Colvin
57b82cf10d
fix: Use IDictionary.GetEnumerator explicitly to iterate keyvals
...
We really need to replace these old non-generic collections with generic ones.
BlueRow's IDictionaryEnumerator implementation iterates over values when passed
to a foreach loop, not key value pairs.
2025-11-15 12:38:49 -05:00
John Colvin
f07d83e7bb
refactor: Make CompositeDataStreamFactory class internal
2025-11-15 12:38:49 -05:00
John Colvin
619be37379
feat (ui): Add ConfigServerPicker dialog
...
So we can connect to different config servers without manually editing spark.conf,
or changing command line args. This will run before the Login dialog.
2025-11-15 12:38:49 -05:00
John Colvin
9348106585
refactor: Move ComboBox styles to ComboBoxes.xaml
...
I'm working on a ConfigServerPicker dialog, which will run before the main
application starts. This allows the dialog to use the same dropdown styles as
the main app without the overhead of loading generic.xaml.
2025-11-15 12:38:49 -05:00
John Colvin
760cdc0fba
cleanup: Fix build warnings, remove unused/redundant code
...
* Make a few member vars and fns local to the constructor
* Replace a couple non-generic collections with generic ones
* Fix warnings and clean code in:
- DictionaryExtensions.cs
* Remove ext fns that are part of dotnet now
- JoinNameResolver.cs
- MainForm.cs
cleanup: Remove unused code in ladder
Some props in cs, and StaticResources in xaml were leftover after I combined
the panel price points into a single collection, bound to an ItemsControl.
2025-11-15 12:38:49 -05:00
John Colvin
fcd1b0e799
refactor: Use generic collection types in PrimitiveManager
2025-11-15 12:38:49 -05:00
John Colvin
eb102a5898
cleanup: Remove unused code in PrimitiveManager.cs
2025-11-15 12:38:48 -05:00
John Colvin
fb3262edea
cleanup: Use file-scoped namespace in PrimitiveManager.cs
...
Also sort using statements
2025-11-15 12:38:48 -05:00
John Colvin
6e1e5d4bbf
refactor: Add IDataStreamFactory(s) for spatial arb streams
2025-11-15 12:38:48 -05:00
John Colvin
256f7025a2
tool: Run Spark.exe from its own directory
...
It looks in the working directory for runtime config appsettings.json files.
2025-11-15 12:38:48 -05:00
John Colvin
24892fb1a8
cleanup: Remove unused BooleanExpressionToken Negate
2025-11-15 12:38:48 -05:00
John Colvin
e3c2efcfcb
doc: Comment why MDConsumer must be kept alive
...
The language server suggests removing it because it isn't referenced.
However, it contains unmanaged resources that mustn't be prematurely
garbage collected.
2025-11-15 12:38:48 -05:00
John Colvin
29080795f7
refactor: Rename AddRemoteQueryManager -> AddRemoteQueryStreams
2025-11-15 12:38:48 -05:00
John Colvin
cd291b7a0d
refactor: Hide md client behind interface, create via Dep Injection
2025-11-15 12:38:48 -05:00
John Colvin
8af9d13802
Drop dump-books and dump-securities from spark_api_server cli
...
These were subcommands were out of place. This exe has one purpose: run
the Spark Api Server.
2025-11-15 12:38:48 -05:00
John Colvin
318d8fb275
cleanup: Remove unused MarketDataClient from OrderSample
2025-11-15 12:38:48 -05:00