Commit Graph
12709 Commits
Author SHA1 Message Date
John Colvin df049622cb fix build: Fix Spark build after making config init async 2025-11-19 13:21:48 -05:00
John Colvin 0f0420ba4c refactor: Change Configuration.initialize to be async
Tweak SparkCoreServices to synchronize Configuration.init globally
2025-11-19 13:18:10 -05:00
John Colvin 6c65cabda8 cleanup: Remove unused PubSubConfig props 2025-11-19 13:17:12 -05:00
John Colvin 64ad720ed6 Capture OperationCanceledException (instead of TaskCanceledException)
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).

https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2025-11-19 13:17:12 -05:00
John Colvin 550bf5dea5 Fix dsqa config json 2025-11-19 13:17:10 -05:00
John Colvin c4cbaff350 fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2025-11-19 13:16:06 -05:00
John Colvin 82c99f0eb3 Fix mismatched types in PubSubServerTagStatsStream schema 2025-11-19 13:16:06 -05:00
John Colvin cbf1ef3d89 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-11-19 13:16:04 -05:00
John Colvin a86ddff4be refactor: Decouple ViewPublisher from IPubTableBroker impl 2025-11-19 12:27:23 -05:00
John Colvin f51d0ae973 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2025-11-19 12:27:20 -05:00
John Colvin b9125fb44b refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2025-11-19 12:24:39 -05:00
John Colvin d51e09719d refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2025-11-19 12:24:39 -05:00
John Colvin 520f6311c7 refactor: Only support 1 tag type in IPubSubServerTagHandler 2025-11-19 12:24:39 -05:00
John Colvin a19658c1e3 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2025-11-19 12:24:39 -05:00
John Colvin 22b46b38ee fix: Dispose market data sub in various md streams 2025-11-19 12:24:38 -05:00
John Colvin c66511a089 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2025-11-19 12:24:38 -05:00
John Colvin b2e94db451 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2025-11-19 12:24:38 -05:00
John Colvin 010f46a2a0 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2025-11-19 12:24:38 -05:00
John Colvin b4f59fd13b chore: Fix typos in log msgs
* Bad string interopolation syntax
2025-11-19 12:24:38 -05:00
John Colvin d5840bf0cc Add spark service modules designed for Dependency Injection 2025-11-19 12:24:38 -05:00
John Colvin a569fec6cc fix build (warn): Only use Registry on windows 2025-11-19 12:24:38 -05:00
John Colvin 77c616af3e cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-11-19 12:24:38 -05:00
John Colvin 4d0d7cf279 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-11-19 12:24:37 -05:00
John Colvin a81a4772d3 cleanup: Modernize PubSubRoute code, json parsing 2025-11-19 12:24:37 -05:00
John Colvin 24114b067b cleanup: Remove unused DateTime grammar from exp parser 2025-11-19 12:24:37 -05:00
John Colvin 57e0f0e8bd refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2025-11-19 12:24:37 -05:00
John Colvin 8fed31c86d chore: Add JsonRpc types for websocket paradigm 2025-11-19 12:24:37 -05:00
John Colvin b04b9db005 cleanup: Remove empty file (clearly added by accident) 2025-11-19 12:24:37 -05:00
John Colvin f571b4992f Add generic ViewStream, BlueRow types/interfaces 2025-11-19 12:24:37 -05:00
John Colvin 1d70eecb44 Add BlueRowColumnMask to hide columns w/ minimal overhead
Less overhead than ProjectStream
2025-11-19 12:24:36 -05:00
John Colvin ca82b8f001 fix: Remove query from literal error stream
The parser can't handle colons in the expression (it throws/crashes)
2025-11-19 12:24:36 -05:00
John Colvin c5df486cf4 fix: Remove erroneous "DSL: " prefix in ai prompt stream
This causes parse errors since we extended the dsl syntax to support
named params in the form of "name: arg".
2025-11-19 12:24:36 -05:00
John Colvin 69632d7327 fix Impl IDictionary<int, IBlueRow> in BlueTable 2025-11-19 12:24:36 -05:00
John Colvin ae21ae02d2 feat: Extend dsl to support explicit param names
When using explicit names, the argument order doesn't matter, and args
with defaults are automatically populated if omitted.
2025-11-19 12:24:36 -05:00
John Colvin 706d1b8de4 Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection 2025-11-19 12:24:36 -05:00
John Colvin eb30e16ef6 ui: Add long/short avg cost lines to ladder 2025-11-19 12:24:36 -05:00
John Colvin acb86b2658 ui: Add exec fee columns to fills grid 2025-11-19 12:24:36 -05:00
John Colvin 05a9d87773 chore: Add exec fee to sec metadata config 2025-11-19 12:24:36 -05:00
John Colvin 8c6dfa4019 chore: Add fill_aggressor prop to ITradeLogRow 2025-11-19 12:24:35 -05:00
John Colvin 3a3c3d45d3 cleanup: Change default DSQA sparkd config server to 127.0.0.1 2025-11-19 10:33:31 -05:00
John Colvin 2014931653 feat: Add live bbo stream 2025-11-19 10:33:31 -05:00
John Colvin aa740e786c refactor: Automatically add id column in BlueTableSchemaBuilder 2025-11-19 00:36:56 -05:00
John Colvin 12c6b367f4 fix (preemptive): Avoid StackOverflow in md client securities fn
One of the 'securities_incremental' fns was calling back into itself instead of
the other overload. This is a 'preemptive' fix because the problematic code path
wasn't running in practice (though it was a ticking time bomb).
2025-11-19 00:36:56 -05:00
John Colvin 5080960b97 refactor: Replace DataPacket w/ IDataPacket in stream handlers
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-19 00:36:56 -05:00
John Colvin 770e14752a chore: Add json facilities for DataStream websocket api 2025-11-19 00:36:56 -05:00
John Colvin e24b877e58 Add gen purpose blue row/table utils for data streams
* Add BlueRowReflection<T> so ViewStream implementations can use
  strong row types without needing to implement IBlueRow
* Add default implementations for IDictionary members in IBlueRow to make
  implementing IBlueRow less tedious
2025-11-19 00:31:53 -05:00
Taha Afzal 09abe4a432 Added RowBasedAppendStream 2025-11-18 03:10:30 +00:00
Taha Afzal 66c0baa150 Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln 2025-09-24 13:16:28 +00:00
arham 5305b32c04 Fix file path didnt work in linux 2025-09-17 04:28:25 +00:00
arham e6304609f8 changes to make streams run on linux 2025-09-16 15:38:21 +00:00