John Colvin
df049622cb
fix build: Fix Spark build after making config init async
2025-11-19 13:21:48 -05:00
John Colvin
0f0420ba4c
refactor: Change Configuration.initialize to be async
...
Tweak SparkCoreServices to synchronize Configuration.init globally
2025-11-19 13:18:10 -05:00
John Colvin
6c65cabda8
cleanup: Remove unused PubSubConfig props
2025-11-19 13:17:12 -05:00
John Colvin
64ad720ed6
Capture OperationCanceledException (instead of TaskCanceledException)
...
The former is the base class for the latter, so this will widen the umbrella to catch a few corner cases
that would otherwise been handled by the generic "Exception" catch block. One corner case I know
of is when an op is canceled via "CancellationToken.ThrowIfCancellationRequested". In that case,
an OperationCanceledException is thrown (not a TaskCanceledException).
https://stackoverflow.com/questions/34359129/operationcanceledexception-vs-taskcanceledexception-when-task-is-canceled
2025-11-19 13:17:12 -05:00
John Colvin
550bf5dea5
Fix dsqa config json
2025-11-19 13:17:10 -05:00
John Colvin
c4cbaff350
fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop
2025-11-19 13:16:06 -05:00
John Colvin
82c99f0eb3
Fix mismatched types in PubSubServerTagStatsStream schema
2025-11-19 13:16:06 -05:00
John Colvin
cbf1ef3d89
Add DataStreamPublisher service modules designed for Dependency Injection
...
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-11-19 13:16:04 -05:00
John Colvin
a86ddff4be
refactor: Decouple ViewPublisher from IPubTableBroker impl
2025-11-19 12:27:23 -05:00
John Colvin
f51d0ae973
Add PubSub service modules designed for Dependency Injection
...
* Get rid of JsonPropertyName attributes in PubSubConfig
- The Dependency Injection framework (based around dotnet ApplicationBuilder)
does not support this attribute, and maps property names verbatim to json fields.
* Change PubSubServer json fields to match cs class property names verbatim
2025-11-19 12:27:20 -05:00
John Colvin
b9125fb44b
refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl
2025-11-19 12:24:39 -05:00
John Colvin
d51e09719d
refactor: Decouple ManagedSubscriber from ISubscriberCore impl
2025-11-19 12:24:39 -05:00
John Colvin
520f6311c7
refactor: Only support 1 tag type in IPubSubServerTagHandler
2025-11-19 12:24:39 -05:00
John Colvin
a19658c1e3
refactor: Decouple PubTableBroker from IPublisherCore impl
...
Reference interface rather than the PublisherCore impl.
2025-11-19 12:24:39 -05:00
John Colvin
22b46b38ee
fix: Dispose market data sub in various md streams
2025-11-19 12:24:38 -05:00
John Colvin
c66511a089
cleanup: Remove unused pubsub tag handler code
...
* Removed 'IPubSubServerTagHandler.register'
- All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
- This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
- No reason these interfaces have to be tied together
2025-11-19 12:24:38 -05:00
John Colvin
b2e94db451
refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
...
This matches the file name
2025-11-19 12:24:38 -05:00
John Colvin
010f46a2a0
Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
...
In config json files (not actual code change)
2025-11-19 12:24:38 -05:00
John Colvin
b4f59fd13b
chore: Fix typos in log msgs
...
* Bad string interopolation syntax
2025-11-19 12:24:38 -05:00
John Colvin
d5840bf0cc
Add spark service modules designed for Dependency Injection
2025-11-19 12:24:38 -05:00
John Colvin
a569fec6cc
fix build (warn): Only use Registry on windows
2025-11-19 12:24:38 -05:00
John Colvin
77c616af3e
cleanup: Fix numerous build warnings in DataStreamSuite
...
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.
* PubSubClientMessage
- Normalize indentation/whitespace
- Fix some typos
- Address nullability issues to fix build warnings
- Modernize code
* Compression related files
- ZipEntry
- ZipFile
- ZipInputStream
- TarHeader
* RemoteMessage
- make 'message' nullable to resolve warnings elsewhere (in calling code)
* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-11-19 12:24:38 -05:00
John Colvin
4d0d7cf279
cleanup: Disable nullability warnings
...
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-11-19 12:24:37 -05:00
John Colvin
a81a4772d3
cleanup: Modernize PubSubRoute code, json parsing
2025-11-19 12:24:37 -05:00
John Colvin
24114b067b
cleanup: Remove unused DateTime grammar from exp parser
2025-11-19 12:24:37 -05:00
John Colvin
57e0f0e8bd
refactor: Add handy WebsocketExtensions
...
* IsSocketStateOpen
* IsClosedProperly
2025-11-19 12:24:37 -05:00
John Colvin
8fed31c86d
chore: Add JsonRpc types for websocket paradigm
2025-11-19 12:24:37 -05:00
John Colvin
b04b9db005
cleanup: Remove empty file (clearly added by accident)
2025-11-19 12:24:37 -05:00
John Colvin
f571b4992f
Add generic ViewStream, BlueRow types/interfaces
2025-11-19 12:24:37 -05:00
John Colvin
1d70eecb44
Add BlueRowColumnMask to hide columns w/ minimal overhead
...
Less overhead than ProjectStream
2025-11-19 12:24:36 -05:00
John Colvin
ca82b8f001
fix: Remove query from literal error stream
...
The parser can't handle colons in the expression (it throws/crashes)
2025-11-19 12:24:36 -05:00
John Colvin
c5df486cf4
fix: Remove erroneous "DSL: " prefix in ai prompt stream
...
This causes parse errors since we extended the dsl syntax to support
named params in the form of "name: arg".
2025-11-19 12:24:36 -05:00
John Colvin
69632d7327
fix Impl IDictionary<int, IBlueRow> in BlueTable
2025-11-19 12:24:36 -05:00
John Colvin
ae21ae02d2
feat: Extend dsl to support explicit param names
...
When using explicit names, the argument order doesn't matter, and args
with defaults are automatically populated if omitted.
2025-11-19 12:24:36 -05:00
John Colvin
706d1b8de4
Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection
2025-11-19 12:24:36 -05:00
John Colvin
eb30e16ef6
ui: Add long/short avg cost lines to ladder
2025-11-19 12:24:36 -05:00
John Colvin
acb86b2658
ui: Add exec fee columns to fills grid
2025-11-19 12:24:36 -05:00
John Colvin
05a9d87773
chore: Add exec fee to sec metadata config
2025-11-19 12:24:36 -05:00
John Colvin
8c6dfa4019
chore: Add fill_aggressor prop to ITradeLogRow
2025-11-19 12:24:35 -05:00
John Colvin
3a3c3d45d3
cleanup: Change default DSQA sparkd config server to 127.0.0.1
2025-11-19 10:33:31 -05:00
John Colvin
2014931653
feat: Add live bbo stream
2025-11-19 10:33:31 -05:00
John Colvin
aa740e786c
refactor: Automatically add id column in BlueTableSchemaBuilder
2025-11-19 00:36:56 -05:00
John Colvin
12c6b367f4
fix (preemptive): Avoid StackOverflow in md client securities fn
...
One of the 'securities_incremental' fns was calling back into itself instead of
the other overload. This is a 'preemptive' fix because the problematic code path
wasn't running in practice (though it was a ticking time bomb).
2025-11-19 00:36:56 -05:00
John Colvin
5080960b97
refactor: Replace DataPacket w/ IDataPacket in stream handlers
...
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-19 00:36:56 -05:00
John Colvin
770e14752a
chore: Add json facilities for DataStream websocket api
2025-11-19 00:36:56 -05:00
John Colvin
e24b877e58
Add gen purpose blue row/table utils for data streams
...
* Add BlueRowReflection<T> so ViewStream implementations can use
strong row types without needing to implement IBlueRow
* Add default implementations for IDictionary members in IBlueRow to make
implementing IBlueRow less tedious
2025-11-19 00:31:53 -05:00
Taha Afzal
09abe4a432
Added RowBasedAppendStream
2025-11-18 03:10:30 +00:00
Taha Afzal
66c0baa150
Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln
2025-09-24 13:16:28 +00:00
arham
5305b32c04
Fix file path didnt work in linux
2025-09-17 04:28:25 +00:00
arham
e6304609f8
changes to make streams run on linux
2025-09-16 15:38:21 +00:00