Commit Graph
14415 Commits
Author SHA1 Message Date
John Colvin bb29c32059 refactor: Rename AddRemoteQueryManager -> AddRemoteQueryStreams 2025-11-19 14:01:00 -05:00
John Colvin a358fe6229 refactor: Hide md client behind interface, create via Dep Injection 2025-11-19 14:00:58 -05:00
John Colvin f7ce346925 Drop dump-books and dump-securities from spark_api_server cli
These were subcommands were out of place. This exe has one purpose: run
the Spark Api Server.
2025-11-19 13:57:48 -05:00
John Colvin e7129b1513 cleanup: Remove unused MarketDataClient from OrderSample 2025-11-19 13:57:48 -05:00
John Colvin 27ee09ade8 build: Launch console for Spark Debug builds, otherwise hide it 2025-11-19 13:57:47 -05:00
John Colvin 63a64af46b cleanup: Remove ancient winforms sdk ui example 2025-11-19 13:57:47 -05:00
John Colvin a4efaa0089 cleanup: Fix a ton of build warnings (cs)
All these warnings have been bothering me for a while. The work isn't done, but
this commit chops them down from ~1000 to ~600.

* AppendCollection.cs
* AppendStreamModified.cs
* BenzingaNewsStream.cs
* BenzingaNewsSymbolStateStream.cs
* BookStream.cs
* CSVSaverStream.cs
* ChartsUtils.cs
* ColumnInfo.cs
* ContractUtils.cs
* ContractUtils.cs
* DiffStream.cs
* FuturesActiveContractStream.cs
* FuzzyJoinOverTimeStream.cs
* HistoricalBarsStream.cs
* HistoricalBarsStream.cs
* InstrumentsStream.cs
* JoinOperandRowIndex.cs
* JoinRow.cs
* JoinStream.cs
* JoinTable.cs
* JoinTableSchema.cs
* LatencyStream.cs
* LeftInternalJoinStream.cs
* LightForeignJoinRow.cs
* LightForeignJoinRowList.cs
* LightLeftForeignJoinRow.cs
* LiteralStream.cs
* NLStream.cs
* NLToDSLStream.cs
* NLToIndexedFewShotPromptStream.cs
* NonIncGroupByStream.cs
* NonIncJoinBookkeeper.cs
* NonIncJoinSpec.cs
* OHLCVStream.cs
* OpenAIPromptStream.cs
* PriceStatsStream.cs
* PriceStatsStream.cs
* ProjectHelper.cs
* ProjectStream.cs
* PubSubServerStateStream.cs
* PubsubHeartbeatStream.cs
* SimpleNode.cs
* SortRow.cs
* SpatialArbStatsStream.cs
* SpatialFuturePairsStream.cs
* SpreadPerformanceEvaluator.cs
* SpreadPerformanceStream.cs
* StreamIOCommon.cs
* SuperJoinStream.cs
* TableMetaDataStream.cs
* TextWriterStream.cs
* TimeAverageStream.cs
2025-11-19 13:57:47 -05:00
John Colvin 7418f5b5a1 style: Normalize indentation in PubsubHeartbeatStream.cs 2025-11-19 13:57:47 -05:00
John Colvin b854390c85 chore: Add GetValue, TryGetValue ext fns to IBlueRow 2025-11-19 13:57:47 -05:00
John Colvin a2f9359077 chore: Add 'Empty' fn for BlueTable/Schema (for convenience) 2025-11-19 13:57:47 -05:00
John Colvin c2db246ce5 cleanup: Remove redundant code block in HistoricalBarsStream.cs 2025-11-19 13:57:47 -05:00
John Colvin 961f55388a cleanup: Fix indentation and build warnings in TestStream.cs 2025-11-19 13:57:47 -05:00
John Colvin 360e80b3fe cleanup: Make MDConsumer.on_security nonnull
It's always initialized in the constructor.
2025-11-19 13:57:47 -05:00
John Colvin 5096912fbf refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore 2025-11-19 13:57:44 -05:00
John Colvin 46e853a55f check pt: PingPong algo
This is a work-in-progress; putting a pin in it for now to focus on
other priorities.
2025-11-19 13:55:44 -05:00
John Colvin 72adf58889 Add environment variable override for app data path
Historically, ~/sparkdata on linux, and ~/AppData/Roaming/bts on win.
Now that we may run multiple exes at once (api server, pubsub, sparkd, etc.),
we need to support multiple data dirs on the same host.
2025-11-19 13:55:43 -05:00
John Colvin f334955acf Drop ".exe" from log dir name on windows 2025-11-19 13:55:43 -05:00
John Colvin d1ee1e2798 feat: Add SpreadBook stream 2025-11-19 13:55:43 -05:00
John Colvin 152ea18266 fix: Check settings Subject.IsDisposed before using 2025-11-19 13:55:43 -05:00
John Colvin ed897c4969 ui (launcher): Move "X" button to left of report menu items
This is a more natural location. Also, it looked weird on the right because the
menu labels aren't fixed width, so the buttons weren't vertically aligned.
2025-11-19 13:55:43 -05:00
John Colvin 4397df4628 refactor: Use HostBuilder to launch spark w/ streams 2025-11-19 13:55:43 -05:00
John Colvin e2b3e543ea build (cleanup): Get rid of global.json
We don't need it anymore
2025-11-19 13:55:43 -05:00
John Colvin 92064bfb48 build: Update System.Composition pkg refs to 8.0.0 2025-11-19 13:55:43 -05:00
John Colvin 3c7f56b0c9 cleanup: Mark a few classes "internal" in Spark 2025-11-19 13:55:43 -05:00
John Colvin cd121defc5 cleanup: Use consistent access modifier in TabDocument partial class
It should be marked internal in both xaml and xaml.cs (it was previously
only so in the latter).
2025-11-19 13:55:43 -05:00
John Colvin ca9a6527ee refactor: Consolidate ShellBootstrapper and base class into one 2025-11-19 13:55:43 -05:00
John Colvin 6cf1b02a75 feat: Add Stream Reports shell window 2025-11-19 13:55:43 -05:00
John Colvin e7ee0e29cc build: Define custom "main" (entry point) for Spark exe
This is a prelude to integrating streams into spark. The plan is to use
an ms HostBuilder to instantiate stream services/dependencies alongside
the spark Application object.
2025-11-19 13:55:43 -05:00
John Colvin 3e28003d80 cleanup: Get rid of stunnel code
This looks like it hasn't been hooked up for years...it would only start
if a stunnel.conf file was detected, which hasn't been around for a very
long time.
2025-11-19 13:55:42 -05:00
John Colvin 63f74222ad fix: Only create mutex if spark app is NOT running in vs designer
I accidentally droped the '!' in a cleanup commit 4 years ago
2025-11-19 13:55:42 -05:00
John Colvin 2ccbfee087 vscode: Add DSQA debug cfg to skip build 2025-11-19 13:55:42 -05:00
John Colvin 2f73a79a6b fix (dsqa): Resolve name conflicts with new BlueOrb streams 2025-11-19 13:55:42 -05:00
John Colvin 4f00d4a20c fix: Fix typo in "DateTimeExpansionStream" type name
Previously, it was "DateTimeExpanstion" (erroneous "t")
2025-11-19 13:55:42 -05:00
John Colvin 8e4d6f9ae3 chore: Add CircularBuffer class (c#) 2025-11-19 13:55:42 -05:00
John Colvin a2b3a4e05d refactor: Rename ExceptionAssert.is_valid ExceptionAssert.Index.IsValid
Like the ExceptionAssert.Argument class that provides methods that throw
ArgumentException if a condition isn't satisfied, the ExceptionAssert.Index
class provides methods that throw IndexOutOfRange exceptions.
2025-11-19 13:55:42 -05:00
John Colvin 47dacc5d1a chore: Add OffsetList class
This supports mutations (unlike the SubRangeWrapper class).
2025-11-19 13:55:42 -05:00
John Colvin 9e842fd453 cleanup: Remove "partial" class modifiers from OrderBook vm classes
They simply weren't necessary.
2025-11-19 13:55:42 -05:00
John Colvin fd98be8f03 check pt: Proof of concept ladder left panel price point collection 2025-11-19 13:55:42 -05:00
John Colvin 43388affc3 ui: Adjust ladder ticks-per-row with ctrl+shift+mousewheel 2025-11-19 13:55:42 -05:00
John Colvin 3e5e6fd487 build: Change publish_data_stream.py to publish the sln
We were publishing individual projects as a workaround for the
dependency issue (bug) that caused some projects to incorrectly target
net8.0. The fact that publishing individual projects "just worked" was
probably dumb luck. Now that the underlying issue has been addressed, we
can resume publishing the sln as a whole.
2025-11-19 13:55:42 -05:00
John Colvin 38daf3fe80 build: Add scripts to publish data stream apps 2025-11-19 13:55:42 -05:00
John Colvin 24040768a2 build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2025-11-19 13:55:42 -05:00
John Colvin 78138eff26 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2025-11-19 13:55:41 -05:00
John Colvin f7c9ee7f6a Reduce log noise in AsyncTCPConnection_AsyncWrapper 2025-11-19 13:55:41 -05:00
John Colvin 054b819eb9 TEMP changes to appsettings.json (at least i should review them before making permanent) 2025-11-19 13:55:41 -05:00
John Colvin 8787c6752b Add ColumnIndex class to resolve column names to indices
Improves performance compared to accessing columns by name
2025-11-19 13:55:41 -05:00
John Colvin 5cd390d78a Increase SQLiteEnumerator page size to 65536
This performs better, and doesn't seem to be too large in practice
2025-11-19 13:55:41 -05:00
John Colvin 8804dbe093 Serialize TimeSpan w/ microsecond precision in DataPacket json 2025-11-19 13:55:41 -05:00
John Colvin fd24abc085 Format DateTimes w/ microsecond precision in DataPacketJsonConverter 2025-11-19 13:55:41 -05:00
John Colvin dce7e73d84 Add BlueColumnTable for pandas integration
This will make it easier/more natural to support pandas column expressions
2025-11-19 13:55:41 -05:00