Commit Graph
14520 Commits
Author SHA1 Message Date
John Colvin 8f53ff5ff9 vscode: Launch DSQA with config via DOTNET_ENVIRONMENT env var 2025-11-15 12:46:26 -05:00
John Colvin 1178aaf410 Add appsettings.Local.json to gitignore (DataStreamSuite) 2025-11-15 12:46:26 -05:00
John Colvin 62b790a156 feat: Add modulus operator to stream DSL 2025-11-15 12:46:25 -05:00
John Colvin 1fc77869e9 chore: Add a few DataStreamExtension fns
* AsDataPackets
  - converts IObservable<IBlueRowCollection> to IObservable<IDataPacket>
* Filter
* OrderBy
* Project
* TimeSeries
* SelectRow - transforms row
* SelectColumn - transforms column
2025-11-15 12:46:25 -05:00
John Colvin 3b91a697e9 improve: Support arbitrary columns in py table_formatter
Not just key/values
2025-11-15 12:46:25 -05:00
John Colvin 8908cc300a chore: Add IDataStreamFactory for FilterStream 2025-11-15 12:46:25 -05:00
John Colvin 5d039b54f5 More jupyter/pandas analysis for Brandon Chen 2025-11-15 12:46:25 -05:00
John Colvin 21f044cc3b refactor: Map TimeSpan to TypeCode.Int64 2025-11-15 12:46:25 -05:00
John Colvin 55c132a033 chore: Add OnWorkScheduler ext to recv IDataPackets on scheduler/thread 2025-11-15 12:46:25 -05:00
John Colvin ddd0de8eca perf: Reset grid when the schema changes in DSQA (not every update) 2025-11-15 12:46:25 -05:00
John Colvin f4451ad2f4 cleanup: Remove commented out code from DSQA MainForm.cs 2025-11-15 12:46:25 -05:00
John Colvin 12d9185b6e fix: Dispose last stream sub before evaluting next expression in DSQA
If the expression throws, we don't want the old stream to keep ticking.
2025-11-15 12:46:25 -05:00
John Colvin 84c3759d17 fix: Add null check to BlueDataPropertyDescriptor
So DSQA doesn't crash on a bad query/table impl. Mainly useful during development
when the stream might have a null row.
2025-11-15 12:46:25 -05:00
John Colvin 618a18b927 refactor: Automatically add id column in BlueTableSchemaBuilder 2025-11-15 12:46:25 -05:00
John Colvin 1e25099c62 cleanup: Enable nullable ref types in IBlueRowCollection.cs 2025-11-15 12:46:25 -05:00
John Colvin afeb5babd4 improve: Support enum types in ColumnInfo 2025-11-15 12:46:25 -05:00
John Colvin 0ff372a63e refactor: Add default impl for columnInfo indexers in IBlueTableSchema 2025-11-15 12:46:25 -05:00
John Colvin 197a81443c chore: Add SumTable (non cummulative)
This is more analgous to the DiffTable, as it computes the sum for
2 rows separated a fixed number of indices.
2025-11-15 12:46:25 -05:00
John Colvin ef98a7eda6 chore: Add prompt examples for BrandonChen 2025-11-15 12:46:24 -05:00
John Colvin 94e9c8c3f2 improve: Generate MLP at runtime with list of example files
Adds prompt file list to NaturalLanguageStreamFactoryConfig
2025-11-15 12:46:24 -05:00
John Colvin 0f9e515d01 chore: Add TableDropIndexEvaluator 2025-11-15 12:46:24 -05:00
John Colvin 342ecd9a52 chore: Add SelectorStream/Table 2025-11-15 12:46:24 -05:00
John Colvin d0c316b3fc chore: Add IndexStream/Table and DropIndex ext fn similar to pandas 2025-11-15 12:46:24 -05:00
John Colvin 44db6190a1 fix: Make PythonDataframeStream thread-safe
Input/output happen on different threads
2025-11-15 12:46:24 -05:00
John Colvin 9856e10eb6 fix: Support some mutations with convert_back delegate in SelectorList 2025-11-15 12:46:24 -05:00
John Colvin deda37b057 chore: Add arithmetic operators to PrimitiveVariant 2025-11-15 12:46:24 -05:00
John Colvin ce5095523b feat: Add a bunch of stream/table ext evaluators 2025-11-15 12:46:24 -05:00
John Colvin 7d374346f2 cleanup: Add NotNull attr to "assert_value" nullable ext 2025-11-15 12:46:24 -05:00
John Colvin ad0e9fa3c9 chore: Add BookDelta class 2025-11-15 12:46:24 -05:00
John Colvin befa17d29d chore: Add DoesNotReturnIf(false) attr to unit test Assert condition 2025-11-15 12:46:24 -05:00
John Colvin 3165ba1235 Add some half-baked Brandon Chen streams 2025-11-15 12:46:23 -05:00
John Colvin 53a9e6a62d fix: Support numeric conversions in ArgumentComparer 2025-11-15 12:46:23 -05:00
John Colvin ff816befb0 chore: Add MergeSorted ext fn to merge 2 already sorted enumerables 2025-11-15 12:46:23 -05:00
John Colvin a69c79b6a9 Add TableUpdate 'WithHistory' ext fn
A buffer of the last N updates is included with each update
2025-11-15 12:46:23 -05:00
John Colvin 7219ff5a95 Show IDataStreamFactory operators in DSQA 2025-11-15 12:46:23 -05:00
John Colvin 603f01699c Tweak databento_client.py 2025-11-15 12:46:23 -05:00
John Colvin 0ca318b222 refactor: Launch ThrottleStream via IDataStreamFactory
Not PrimitiveManager
2025-11-15 12:45:08 -05:00
John Colvin 4addb58743 feat: Add BboPriceSummaryStream for Brandon Chen 2025-11-15 12:45:08 -05:00
John Colvin c365330b08 feat: Add GroupByApplyStream 2025-11-15 12:45:08 -05:00
John Colvin 4bb4008ac6 feat: Add SliceTable/Stream 2025-11-15 12:45:07 -05:00
John Colvin 001892ab34 Jupyter/pandas analysis for brandon chen 2025-11-15 12:45:07 -05:00
John Colvin e77ee859a3 fix: Preserve clipboard (copy/paste) when DSQA exits 2025-11-15 12:45:07 -05:00
John Colvin c1a8702a47 ui: Improve dateTime formatting in DSQA 2025-11-15 12:45:07 -05:00
John Colvin 6045c602dc feat: Add pseudo member (extension) function calls to DSL 2025-11-15 12:45:07 -05:00
John Colvin 4a2a814af0 refactor: Move StreamQueryMetadata into StreamFactories namespace 2025-11-15 12:45:07 -05:00
John Colvin 08a655b1d5 Change WellKnownColumn names to match those in mlp
This is easier than changing them everywhere in the master learning prompt.
2025-11-15 12:45:07 -05:00
John Colvin 2830af5644 improve: Add config for natural language streams
They're created by a stream factory now, which has the config handy.
Config includes api key, model name, mlp file, etc.

fix add config for natural language streams
2025-11-15 12:45:07 -05:00
John Colvin 659eb83f4d refactor: Drop null type modifier in TryGetFirst ext
Replace w/ MaybeNullWhen attributes
2025-11-15 12:45:07 -05:00
John Colvin 5d9d364de3 cleanup: Remove unnecessary code 2025-11-15 12:45:07 -05:00
John Colvin 07c1ad3204 qol: Don't connect to sparkd by default in Spark.Console cli 2025-11-15 12:45:07 -05:00