John Colvin
94e9c8c3f2
improve: Generate MLP at runtime with list of example files
...
Adds prompt file list to NaturalLanguageStreamFactoryConfig
2025-11-15 12:46:24 -05:00
John Colvin
0f9e515d01
chore: Add TableDropIndexEvaluator
2025-11-15 12:46:24 -05:00
John Colvin
342ecd9a52
chore: Add SelectorStream/Table
2025-11-15 12:46:24 -05:00
John Colvin
d0c316b3fc
chore: Add IndexStream/Table and DropIndex ext fn similar to pandas
2025-11-15 12:46:24 -05:00
John Colvin
44db6190a1
fix: Make PythonDataframeStream thread-safe
...
Input/output happen on different threads
2025-11-15 12:46:24 -05:00
John Colvin
9856e10eb6
fix: Support some mutations with convert_back delegate in SelectorList
2025-11-15 12:46:24 -05:00
John Colvin
deda37b057
chore: Add arithmetic operators to PrimitiveVariant
2025-11-15 12:46:24 -05:00
John Colvin
ce5095523b
feat: Add a bunch of stream/table ext evaluators
2025-11-15 12:46:24 -05:00
John Colvin
7d374346f2
cleanup: Add NotNull attr to "assert_value" nullable ext
2025-11-15 12:46:24 -05:00
John Colvin
ad0e9fa3c9
chore: Add BookDelta class
2025-11-15 12:46:24 -05:00
John Colvin
befa17d29d
chore: Add DoesNotReturnIf(false) attr to unit test Assert condition
2025-11-15 12:46:24 -05:00
John Colvin
3165ba1235
Add some half-baked Brandon Chen streams
2025-11-15 12:46:23 -05:00
John Colvin
53a9e6a62d
fix: Support numeric conversions in ArgumentComparer
2025-11-15 12:46:23 -05:00
John Colvin
ff816befb0
chore: Add MergeSorted ext fn to merge 2 already sorted enumerables
2025-11-15 12:46:23 -05:00
John Colvin
a69c79b6a9
Add TableUpdate 'WithHistory' ext fn
...
A buffer of the last N updates is included with each update
2025-11-15 12:46:23 -05:00
John Colvin
7219ff5a95
Show IDataStreamFactory operators in DSQA
2025-11-15 12:46:23 -05:00
John Colvin
603f01699c
Tweak databento_client.py
2025-11-15 12:46:23 -05:00
John Colvin
0ca318b222
refactor: Launch ThrottleStream via IDataStreamFactory
...
Not PrimitiveManager
2025-11-15 12:45:08 -05:00
John Colvin
4addb58743
feat: Add BboPriceSummaryStream for Brandon Chen
2025-11-15 12:45:08 -05:00
John Colvin
c365330b08
feat: Add GroupByApplyStream
2025-11-15 12:45:08 -05:00
John Colvin
4bb4008ac6
feat: Add SliceTable/Stream
2025-11-15 12:45:07 -05:00
John Colvin
001892ab34
Jupyter/pandas analysis for brandon chen
2025-11-15 12:45:07 -05:00
John Colvin
e77ee859a3
fix: Preserve clipboard (copy/paste) when DSQA exits
2025-11-15 12:45:07 -05:00
John Colvin
c1a8702a47
ui: Improve dateTime formatting in DSQA
2025-11-15 12:45:07 -05:00
John Colvin
6045c602dc
feat: Add pseudo member (extension) function calls to DSL
2025-11-15 12:45:07 -05:00
John Colvin
4a2a814af0
refactor: Move StreamQueryMetadata into StreamFactories namespace
2025-11-15 12:45:07 -05:00
John Colvin
08a655b1d5
Change WellKnownColumn names to match those in mlp
...
This is easier than changing them everywhere in the master learning prompt.
2025-11-15 12:45:07 -05:00
John Colvin
2830af5644
improve: Add config for natural language streams
...
They're created by a stream factory now, which has the config handy.
Config includes api key, model name, mlp file, etc.
fix add config for natural language streams
2025-11-15 12:45:07 -05:00
John Colvin
659eb83f4d
refactor: Drop null type modifier in TryGetFirst ext
...
Replace w/ MaybeNullWhen attributes
2025-11-15 12:45:07 -05:00
John Colvin
5d9d364de3
cleanup: Remove unnecessary code
2025-11-15 12:45:07 -05:00
John Colvin
07c1ad3204
qol: Don't connect to sparkd by default in Spark.Console cli
2025-11-15 12:45:07 -05:00
John Colvin
e61829f8e0
qol: Change default pubsub url from 'pubsub' to 127.0.0.1
...
The hostname 'pubsub' works when running pubsub and other services
via docker compose. In practice, it seems we usually just run them on a
single host, without using docker compose.
2025-11-15 12:45:07 -05:00
John Colvin
0cf44fcc40
feat: Add historic market data streams factory
...
Loads historic md from Databento files in structured paths. It is
designed to work with files downloaded via databento_client.py
2025-11-15 12:45:04 -05:00
John Colvin
2549e34a17
Create LiveBBOStream via BookStreamFactory
...
Instead of PrimitiveManager
2025-11-15 12:42:40 -05:00
John Colvin
693f2deb67
fix: Don't modify input env in BoundBooleanExpression.Evaluate
2025-11-15 12:42:40 -05:00
John Colvin
63665e9b10
refactor: Pass IMarketDataClient to streams via dependency injection
...
In general, the streams should not instantiate services, etc. themselves.
2025-11-15 12:42:40 -05:00
John Colvin
a79f5ff860
improve: Support all integer types in IBlueRow Id ext fn
2025-11-15 12:42:40 -05:00
John Colvin
dae75c507e
cleanup: Remove unused code CSharpString.cs
2025-11-15 12:42:40 -05:00
John Colvin
4db9cdd357
feat: Add DiffStream to compute row diffs (like pandas)
2025-11-15 12:42:40 -05:00
John Colvin
18b19ac787
refactor: Rewrite SortStream now that IBlueRowCollection is IList
2025-11-15 12:42:40 -05:00
John Colvin
b5ea918378
refactor: Add param for TimeSeriesStream timestamp col name
...
Default it to 'ts_event' instead of 'timestamp' (this is consistent w/
Databento schema fwiw)
2025-11-15 12:42:40 -05:00
John Colvin
9df4ecf6c8
fix: Trim backticks from nl generated dsl
2025-11-15 12:42:40 -05:00
John Colvin
957f6cb33f
feat: Add FeatherReader stream
...
We don't need this right now, but I already wrote it...it might be
useful later.
2025-11-15 12:42:39 -05:00
John Colvin
03294ffe37
feat: Add function defs to DataStream DSL
2025-11-15 12:42:39 -05:00
John Colvin
0550078de0
chore: Support mutations in AggregateDictionary (optionally)
2025-11-15 12:42:39 -05:00
John Colvin
89129f726e
chore: Implement IComparable in BookTuple
2025-11-15 12:42:39 -05:00
John Colvin
97eefb1a1e
improve: Serialize to/from parquet in PythonDataframeStream
2025-11-15 12:42:39 -05:00
John Colvin
805693e170
feat: Add ParquetReader/Writer streams
2025-11-15 12:40:48 -05:00
John Colvin
3a7f3ac608
fix: Fire multiple rows at once in replay bbo/trade streams doWork fn
...
If necessary to keep pace with timestamps in the file.
2025-11-15 12:38:51 -05:00
John Colvin
b82cc31d2d
refactor: Launch LiveTradeStream via IDataStreamFactory
2025-11-15 12:38:51 -05:00