Commit Graph
14520 Commits
Author SHA1 Message Date
John Colvin 06f4f6623d vscode: Launch DSQA with config via DOTNET_ENVIRONMENT env var 2025-11-19 14:44:56 -05:00
John Colvin 74ce1fc98e Add appsettings.Local.json to gitignore (DataStreamSuite) 2025-11-19 14:44:56 -05:00
John Colvin 3a3352eef4 feat: Add modulus operator to stream DSL 2025-11-19 14:44:56 -05:00
John Colvin 28316c20d2 chore: Add a few DataStreamExtension fns
* AsDataPackets
  - converts IObservable<IBlueRowCollection> to IObservable<IDataPacket>
* Filter
* OrderBy
* Project
* TimeSeries
* SelectRow - transforms row
* SelectColumn - transforms column
2025-11-19 14:44:56 -05:00
John Colvin b171ce2ff0 improve: Support arbitrary columns in py table_formatter
Not just key/values
2025-11-19 14:44:55 -05:00
John Colvin 3431700c38 chore: Add IDataStreamFactory for FilterStream 2025-11-19 14:44:55 -05:00
John Colvin 3638b26bdf More jupyter/pandas analysis for Brandon Chen 2025-11-19 14:44:55 -05:00
John Colvin 9395fe70b3 refactor: Map TimeSpan to TypeCode.Int64 2025-11-19 14:44:55 -05:00
John Colvin ecc524d1ba chore: Add OnWorkScheduler ext to recv IDataPackets on scheduler/thread 2025-11-19 14:44:55 -05:00
John Colvin 680761dabc perf: Reset grid when the schema changes in DSQA (not every update) 2025-11-19 14:44:55 -05:00
John Colvin 96116c7f8f cleanup: Remove commented out code from DSQA MainForm.cs 2025-11-19 14:44:55 -05:00
John Colvin 23c20bb255 fix: Dispose last stream sub before evaluting next expression in DSQA
If the expression throws, we don't want the old stream to keep ticking.
2025-11-19 14:44:55 -05:00
John Colvin f86795d18d fix: Add null check to BlueDataPropertyDescriptor
So DSQA doesn't crash on a bad query/table impl. Mainly useful during development
when the stream might have a null row.
2025-11-19 14:44:55 -05:00
John Colvin d7d3c43302 cleanup: Enable nullable ref types in IBlueRowCollection.cs 2025-11-19 14:44:55 -05:00
John Colvin 71bd231a4a improve: Support enum types in ColumnInfo 2025-11-19 14:44:55 -05:00
John Colvin 7ecf705f53 refactor: Add default impl for columnInfo indexers in IBlueTableSchema 2025-11-19 14:44:55 -05:00
John Colvin 8f3a3fa3ec chore: Add SumTable (non cummulative)
This is more analgous to the DiffTable, as it computes the sum for
2 rows separated a fixed number of indices.
2025-11-19 14:44:55 -05:00
John Colvin 30ee997040 chore: Add prompt examples for BrandonChen 2025-11-19 14:44:54 -05:00
John Colvin 87307f226d improve: Generate MLP at runtime with list of example files
Adds prompt file list to NaturalLanguageStreamFactoryConfig
2025-11-19 14:44:54 -05:00
John Colvin c8d52e0577 chore: Add TableDropIndexEvaluator 2025-11-19 14:44:54 -05:00
John Colvin a0511639f9 chore: Add SelectorStream/Table 2025-11-19 14:44:54 -05:00
John Colvin e569d8cc1d chore: Add IndexStream/Table and DropIndex ext fn similar to pandas 2025-11-19 14:44:54 -05:00
John Colvin 6d58fb76a2 fix: Make PythonDataframeStream thread-safe
Input/output happen on different threads
2025-11-19 14:44:54 -05:00
John Colvin c2409d3edd fix: Support some mutations with convert_back delegate in SelectorList 2025-11-19 14:44:54 -05:00
John Colvin 2934346769 chore: Add arithmetic operators to PrimitiveVariant 2025-11-19 14:44:54 -05:00
John Colvin f00bb309e1 feat: Add a bunch of stream/table ext evaluators 2025-11-19 14:44:54 -05:00
John Colvin 2c8b2b6918 cleanup: Add NotNull attr to "assert_value" nullable ext 2025-11-19 14:44:54 -05:00
John Colvin c1cf1eab0a chore: Add BookDelta class 2025-11-19 14:44:54 -05:00
John Colvin af248ddd69 chore: Add DoesNotReturnIf(false) attr to unit test Assert condition 2025-11-19 14:44:53 -05:00
John Colvin 0d33b19a39 Add some half-baked Brandon Chen streams 2025-11-19 14:44:53 -05:00
John Colvin 5665d18249 fix: Support numeric conversions in ArgumentComparer 2025-11-19 14:44:53 -05:00
John Colvin a3615f1d7e chore: Add MergeSorted ext fn to merge 2 already sorted enumerables 2025-11-19 14:44:53 -05:00
John Colvin 7242c87e0f Add TableUpdate 'WithHistory' ext fn
A buffer of the last N updates is included with each update
2025-11-19 14:44:53 -05:00
John Colvin 2bd65ad48f Show IDataStreamFactory operators in DSQA 2025-11-19 14:44:53 -05:00
John Colvin 08163ca6c0 Tweak databento_client.py 2025-11-19 14:44:53 -05:00
John Colvin 1640e3854c refactor: Launch ThrottleStream via IDataStreamFactory
Not PrimitiveManager
2025-11-19 14:44:53 -05:00
John Colvin c4bd606017 feat: Add BboPriceSummaryStream for Brandon Chen 2025-11-19 14:44:50 -05:00
John Colvin 0d75dc011f feat: Add GroupByApplyStream 2025-11-19 14:42:04 -05:00
John Colvin ce455ac52c feat: Add SliceTable/Stream 2025-11-19 14:40:50 -05:00
John Colvin a181bde099 Jupyter/pandas analysis for brandon chen 2025-11-19 14:40:50 -05:00
John Colvin e1b482e374 fix: Preserve clipboard (copy/paste) when DSQA exits 2025-11-19 14:40:50 -05:00
John Colvin e6dd8ccafb ui: Improve dateTime formatting in DSQA 2025-11-19 14:40:50 -05:00
John Colvin 36366cb7c4 feat: Add pseudo member (extension) function calls to DSL 2025-11-19 14:40:49 -05:00
John Colvin dbc5304546 refactor: Move StreamQueryMetadata into StreamFactories namespace 2025-11-19 14:40:49 -05:00
John Colvin b17b8fc7c8 improve: Add config for natural language streams
They're created by a stream factory now, which has the config handy.
Config includes api key, model name, mlp file, etc.

fix add config for natural language streams
2025-11-19 14:40:17 -05:00
John Colvin 7a65758fa5 refactor: Drop null type modifier in TryGetFirst ext
Replace w/ MaybeNullWhen attributes
2025-11-19 14:40:17 -05:00
John Colvin 79aa541bd7 cleanup: Remove unnecessary code 2025-11-19 14:40:17 -05:00
John Colvin f35dae7110 qol: Don't connect to sparkd by default in Spark.Console cli 2025-11-19 14:40:17 -05:00
John Colvin 02290f2f34 qol: Change default pubsub url from 'pubsub' to 127.0.0.1
The hostname 'pubsub' works when running pubsub and other services
via docker compose. In practice, it seems we usually just run them on a
single host, without using docker compose.
2025-11-19 14:40:17 -05:00
John Colvin 04200a330a feat: Add historic market data streams factory
Loads historic md from Databento files in structured paths. It is
designed to work with files downloaded via databento_client.py
2025-11-19 14:40:15 -05:00