John Colvin
8f53ff5ff9
vscode: Launch DSQA with config via DOTNET_ENVIRONMENT env var
2025-11-15 12:46:26 -05:00
John Colvin
1178aaf410
Add appsettings.Local.json to gitignore (DataStreamSuite)
2025-11-15 12:46:26 -05:00
John Colvin
62b790a156
feat: Add modulus operator to stream DSL
2025-11-15 12:46:25 -05:00
John Colvin
1fc77869e9
chore: Add a few DataStreamExtension fns
...
* AsDataPackets
- converts IObservable<IBlueRowCollection> to IObservable<IDataPacket>
* Filter
* OrderBy
* Project
* TimeSeries
* SelectRow - transforms row
* SelectColumn - transforms column
2025-11-15 12:46:25 -05:00
John Colvin
3b91a697e9
improve: Support arbitrary columns in py table_formatter
...
Not just key/values
2025-11-15 12:46:25 -05:00
John Colvin
8908cc300a
chore: Add IDataStreamFactory for FilterStream
2025-11-15 12:46:25 -05:00
John Colvin
5d039b54f5
More jupyter/pandas analysis for Brandon Chen
2025-11-15 12:46:25 -05:00
John Colvin
21f044cc3b
refactor: Map TimeSpan to TypeCode.Int64
2025-11-15 12:46:25 -05:00
John Colvin
55c132a033
chore: Add OnWorkScheduler ext to recv IDataPackets on scheduler/thread
2025-11-15 12:46:25 -05:00
John Colvin
ddd0de8eca
perf: Reset grid when the schema changes in DSQA (not every update)
2025-11-15 12:46:25 -05:00
John Colvin
f4451ad2f4
cleanup: Remove commented out code from DSQA MainForm.cs
2025-11-15 12:46:25 -05:00
John Colvin
12d9185b6e
fix: Dispose last stream sub before evaluting next expression in DSQA
...
If the expression throws, we don't want the old stream to keep ticking.
2025-11-15 12:46:25 -05:00
John Colvin
84c3759d17
fix: Add null check to BlueDataPropertyDescriptor
...
So DSQA doesn't crash on a bad query/table impl. Mainly useful during development
when the stream might have a null row.
2025-11-15 12:46:25 -05:00
John Colvin
618a18b927
refactor: Automatically add id column in BlueTableSchemaBuilder
2025-11-15 12:46:25 -05:00
John Colvin
1e25099c62
cleanup: Enable nullable ref types in IBlueRowCollection.cs
2025-11-15 12:46:25 -05:00
John Colvin
afeb5babd4
improve: Support enum types in ColumnInfo
2025-11-15 12:46:25 -05:00
John Colvin
0ff372a63e
refactor: Add default impl for columnInfo indexers in IBlueTableSchema
2025-11-15 12:46:25 -05:00
John Colvin
197a81443c
chore: Add SumTable (non cummulative)
...
This is more analgous to the DiffTable, as it computes the sum for
2 rows separated a fixed number of indices.
2025-11-15 12:46:25 -05:00
John Colvin
ef98a7eda6
chore: Add prompt examples for BrandonChen
2025-11-15 12:46:24 -05:00
John Colvin
94e9c8c3f2
improve: Generate MLP at runtime with list of example files
...
Adds prompt file list to NaturalLanguageStreamFactoryConfig
2025-11-15 12:46:24 -05:00
John Colvin
0f9e515d01
chore: Add TableDropIndexEvaluator
2025-11-15 12:46:24 -05:00
John Colvin
342ecd9a52
chore: Add SelectorStream/Table
2025-11-15 12:46:24 -05:00
John Colvin
d0c316b3fc
chore: Add IndexStream/Table and DropIndex ext fn similar to pandas
2025-11-15 12:46:24 -05:00
John Colvin
44db6190a1
fix: Make PythonDataframeStream thread-safe
...
Input/output happen on different threads
2025-11-15 12:46:24 -05:00
John Colvin
9856e10eb6
fix: Support some mutations with convert_back delegate in SelectorList
2025-11-15 12:46:24 -05:00
John Colvin
deda37b057
chore: Add arithmetic operators to PrimitiveVariant
2025-11-15 12:46:24 -05:00
John Colvin
ce5095523b
feat: Add a bunch of stream/table ext evaluators
2025-11-15 12:46:24 -05:00
John Colvin
7d374346f2
cleanup: Add NotNull attr to "assert_value" nullable ext
2025-11-15 12:46:24 -05:00
John Colvin
ad0e9fa3c9
chore: Add BookDelta class
2025-11-15 12:46:24 -05:00
John Colvin
befa17d29d
chore: Add DoesNotReturnIf(false) attr to unit test Assert condition
2025-11-15 12:46:24 -05:00
John Colvin
3165ba1235
Add some half-baked Brandon Chen streams
2025-11-15 12:46:23 -05:00
John Colvin
53a9e6a62d
fix: Support numeric conversions in ArgumentComparer
2025-11-15 12:46:23 -05:00
John Colvin
ff816befb0
chore: Add MergeSorted ext fn to merge 2 already sorted enumerables
2025-11-15 12:46:23 -05:00
John Colvin
a69c79b6a9
Add TableUpdate 'WithHistory' ext fn
...
A buffer of the last N updates is included with each update
2025-11-15 12:46:23 -05:00
John Colvin
7219ff5a95
Show IDataStreamFactory operators in DSQA
2025-11-15 12:46:23 -05:00
John Colvin
603f01699c
Tweak databento_client.py
2025-11-15 12:46:23 -05:00
John Colvin
0ca318b222
refactor: Launch ThrottleStream via IDataStreamFactory
...
Not PrimitiveManager
2025-11-15 12:45:08 -05:00
John Colvin
4addb58743
feat: Add BboPriceSummaryStream for Brandon Chen
2025-11-15 12:45:08 -05:00
John Colvin
c365330b08
feat: Add GroupByApplyStream
2025-11-15 12:45:08 -05:00
John Colvin
4bb4008ac6
feat: Add SliceTable/Stream
2025-11-15 12:45:07 -05:00
John Colvin
001892ab34
Jupyter/pandas analysis for brandon chen
2025-11-15 12:45:07 -05:00
John Colvin
e77ee859a3
fix: Preserve clipboard (copy/paste) when DSQA exits
2025-11-15 12:45:07 -05:00
John Colvin
c1a8702a47
ui: Improve dateTime formatting in DSQA
2025-11-15 12:45:07 -05:00
John Colvin
6045c602dc
feat: Add pseudo member (extension) function calls to DSL
2025-11-15 12:45:07 -05:00
John Colvin
4a2a814af0
refactor: Move StreamQueryMetadata into StreamFactories namespace
2025-11-15 12:45:07 -05:00
John Colvin
08a655b1d5
Change WellKnownColumn names to match those in mlp
...
This is easier than changing them everywhere in the master learning prompt.
2025-11-15 12:45:07 -05:00
John Colvin
2830af5644
improve: Add config for natural language streams
...
They're created by a stream factory now, which has the config handy.
Config includes api key, model name, mlp file, etc.
fix add config for natural language streams
2025-11-15 12:45:07 -05:00
John Colvin
659eb83f4d
refactor: Drop null type modifier in TryGetFirst ext
...
Replace w/ MaybeNullWhen attributes
2025-11-15 12:45:07 -05:00
John Colvin
5d9d364de3
cleanup: Remove unnecessary code
2025-11-15 12:45:07 -05:00
John Colvin
07c1ad3204
qol: Don't connect to sparkd by default in Spark.Console cli
2025-11-15 12:45:07 -05:00