Commit Graph
12881 Commits
Author SHA1 Message Date
John Colvin c8d52e0577 chore: Add TableDropIndexEvaluator 2025-11-19 14:44:54 -05:00
John Colvin a0511639f9 chore: Add SelectorStream/Table 2025-11-19 14:44:54 -05:00
John Colvin e569d8cc1d chore: Add IndexStream/Table and DropIndex ext fn similar to pandas 2025-11-19 14:44:54 -05:00
John Colvin 6d58fb76a2 fix: Make PythonDataframeStream thread-safe
Input/output happen on different threads
2025-11-19 14:44:54 -05:00
John Colvin c2409d3edd fix: Support some mutations with convert_back delegate in SelectorList 2025-11-19 14:44:54 -05:00
John Colvin 2934346769 chore: Add arithmetic operators to PrimitiveVariant 2025-11-19 14:44:54 -05:00
John Colvin f00bb309e1 feat: Add a bunch of stream/table ext evaluators 2025-11-19 14:44:54 -05:00
John Colvin 2c8b2b6918 cleanup: Add NotNull attr to "assert_value" nullable ext 2025-11-19 14:44:54 -05:00
John Colvin c1cf1eab0a chore: Add BookDelta class 2025-11-19 14:44:54 -05:00
John Colvin af248ddd69 chore: Add DoesNotReturnIf(false) attr to unit test Assert condition 2025-11-19 14:44:53 -05:00
John Colvin 0d33b19a39 Add some half-baked Brandon Chen streams 2025-11-19 14:44:53 -05:00
John Colvin 5665d18249 fix: Support numeric conversions in ArgumentComparer 2025-11-19 14:44:53 -05:00
John Colvin a3615f1d7e chore: Add MergeSorted ext fn to merge 2 already sorted enumerables 2025-11-19 14:44:53 -05:00
John Colvin 7242c87e0f Add TableUpdate 'WithHistory' ext fn
A buffer of the last N updates is included with each update
2025-11-19 14:44:53 -05:00
John Colvin 2bd65ad48f Show IDataStreamFactory operators in DSQA 2025-11-19 14:44:53 -05:00
John Colvin 1640e3854c refactor: Launch ThrottleStream via IDataStreamFactory
Not PrimitiveManager
2025-11-19 14:44:53 -05:00
John Colvin c4bd606017 feat: Add BboPriceSummaryStream for Brandon Chen 2025-11-19 14:44:50 -05:00
John Colvin 0d75dc011f feat: Add GroupByApplyStream 2025-11-19 14:42:04 -05:00
John Colvin ce455ac52c feat: Add SliceTable/Stream 2025-11-19 14:40:50 -05:00
John Colvin a181bde099 Jupyter/pandas analysis for brandon chen 2025-11-19 14:40:50 -05:00
John Colvin e1b482e374 fix: Preserve clipboard (copy/paste) when DSQA exits 2025-11-19 14:40:50 -05:00
John Colvin e6dd8ccafb ui: Improve dateTime formatting in DSQA 2025-11-19 14:40:50 -05:00
John Colvin 36366cb7c4 feat: Add pseudo member (extension) function calls to DSL 2025-11-19 14:40:49 -05:00
John Colvin dbc5304546 refactor: Move StreamQueryMetadata into StreamFactories namespace 2025-11-19 14:40:49 -05:00
John Colvin b17b8fc7c8 improve: Add config for natural language streams
They're created by a stream factory now, which has the config handy.
Config includes api key, model name, mlp file, etc.

fix add config for natural language streams
2025-11-19 14:40:17 -05:00
John Colvin 7a65758fa5 refactor: Drop null type modifier in TryGetFirst ext
Replace w/ MaybeNullWhen attributes
2025-11-19 14:40:17 -05:00
John Colvin 79aa541bd7 cleanup: Remove unnecessary code 2025-11-19 14:40:17 -05:00
John Colvin f35dae7110 qol: Don't connect to sparkd by default in Spark.Console cli 2025-11-19 14:40:17 -05:00
John Colvin 02290f2f34 qol: Change default pubsub url from 'pubsub' to 127.0.0.1
The hostname 'pubsub' works when running pubsub and other services
via docker compose. In practice, it seems we usually just run them on a
single host, without using docker compose.
2025-11-19 14:40:17 -05:00
John Colvin 04200a330a feat: Add historic market data streams factory
Loads historic md from Databento files in structured paths. It is
designed to work with files downloaded via databento_client.py
2025-11-19 14:40:15 -05:00
John Colvin 0ea884b505 fix: Don't modify input env in BoundBooleanExpression.Evaluate 2025-11-19 14:35:12 -05:00
John Colvin 2d0df72f46 refactor: Pass IMarketDataClient to streams via dependency injection
In general, the streams should not instantiate services, etc. themselves.
2025-11-19 14:35:12 -05:00
John Colvin 8766786109 improve: Support all integer types in IBlueRow Id ext fn 2025-11-19 14:35:12 -05:00
John Colvin e48eeb4df3 cleanup: Remove unused code CSharpString.cs 2025-11-19 14:35:11 -05:00
John Colvin b1bfc6f65b feat: Add DiffStream to compute row diffs (like pandas) 2025-11-19 14:35:11 -05:00
John Colvin 6f209b97d4 refactor: Rewrite SortStream now that IBlueRowCollection is IList 2025-11-19 14:35:11 -05:00
John Colvin 751980ba67 refactor: Add param for TimeSeriesStream timestamp col name
Default it to 'ts_event' instead of 'timestamp' (this is consistent w/
Databento schema fwiw)
2025-11-19 14:35:11 -05:00
John Colvin 05379f6ac4 fix: Trim backticks from nl generated dsl 2025-11-19 14:35:11 -05:00
John Colvin fc1a89a60f feat: Add FeatherReader stream
We don't need this right now, but I already wrote it...it might be
useful later.
2025-11-19 14:35:11 -05:00
John Colvin 45a9322090 feat: Add function defs to DataStream DSL 2025-11-19 14:35:11 -05:00
John Colvin a0276ac34f chore: Support mutations in AggregateDictionary (optionally) 2025-11-19 14:35:11 -05:00
John Colvin 827079f5f5 chore: Implement IComparable in BookTuple 2025-11-19 14:35:11 -05:00
John Colvin 75f0cd7831 improve: Serialize to/from parquet in PythonDataframeStream 2025-11-19 14:35:08 -05:00
John Colvin 79c3e544aa feat: Add ParquetReader/Writer streams 2025-11-19 14:32:37 -05:00
John Colvin 4aebdad90b fix: Fire multiple rows at once in replay bbo/trade streams doWork fn
If necessary to keep pace with timestamps in the file.
2025-11-19 14:32:34 -05:00
John Colvin d7300cbddb refactor: Launch LiveTradeStream via IDataStreamFactory 2025-11-19 14:27:48 -05:00
John Colvin c21cd2ba6d chore: Add id field to TradeRecord
Need this for streams
2025-11-19 14:27:48 -05:00
John Colvin 488fbe3c43 fix: Actually use sec filter in MarketDataClient.Trades 2025-11-19 14:27:48 -05:00
John Colvin d25db044af style: Rename dispose_thread_safe in PascalCase 2025-11-19 14:27:48 -05:00
John Colvin d3e31e4e44 refactor: Make IBlueRowCollection inherit IList<IBlueRow> 2025-11-19 14:27:46 -05:00