Commit Graph
6295 Commits
Author SHA1 Message Date
John Colvin 3dcde6d089 refactor: Remove spark api server dependency on SecurityMD
This is part of a larger effort to decouple the api server from spark grpc
(protobuf) services
2025-12-05 15:56:23 +00:00
John Colvin 5f4a811598 feat: Add ParquetReader/Writer streams 2025-12-05 15:56:23 +00:00
John Colvin 5fb64e114c refactor: Hide md client behind interface, create via dep injection 2025-12-05 15:56:21 +00:00
John Colvin 0571429375 refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore 2025-12-05 15:54:03 +00:00
John Colvin 5412a90c2e fix: Check settings Subject.IsDisposed before using 2025-12-05 15:54:03 +00:00
John Colvin 50b4cf7556 ui (launcher): Move "X" button to left of report menu items
This is a more natural location. Also, it looked weird on the right because the
menu labels aren't fixed width, so the buttons weren't vertically aligned.
2025-12-05 15:54:03 +00:00
John Colvin 247dcf7d31 refactor: Use HostBuilder to launch spark w/ streams 2025-12-05 15:54:03 +00:00
John Colvin 05d2a54e8a build: Update System.Composition pkg refs to 8.0.0 2025-12-05 15:54:03 +00:00
John Colvin 6b27106f44 cleanup: Mark a few classes "internal" in Spark
* BookmarkShellModule
* TabDocument
  - In xaml only (it was already marked internal in xaml.cs)
* WindowManager
* WorkspaceViewModel
* WorkspacesDialogView
* WorkspacesDialogViewModel
* WorkspacesModel
2025-12-05 15:54:03 +00:00
John Colvin aadce1c989 refactor: Consolidate ShellBootstrapper and base class into one 2025-12-05 15:54:03 +00:00
John Colvin b532836d9c feat: Add Stream Reports shell window 2025-12-05 15:54:03 +00:00
John Colvin 3135778120 build: Define custom "main" (entry point) for Spark exe
This is a prelude to integrating streams into spark. The plan is to use
an ms HostBuilder to instantiate stream services/dependencies alongside
the spark Application object.
2025-12-05 15:54:03 +00:00
John Colvin 26e500368f cleanup: Get rid of stunnel code
This looks like it hasn't been hooked up for years...it would only start
if a stunnel.conf file was detected, which hasn't been around for a very
long time.
2025-12-05 15:54:03 +00:00
John Colvin 8957ef6230 fix: Only create mutex if spark app is NOT running in vs designer
I accidentally droped the '!' in a cleanup commit 4 years ago
2025-12-05 15:54:03 +00:00
John Colvin 8a5e4048f8 cleanup: Remove "partial" class modifiers from OrderBook vm classes
They simply weren't necessary.
2025-12-05 15:54:03 +00:00
John Colvin 3636d1c785 refactor: Rename snake_case fns PascalCase 2025-12-05 15:54:03 +00:00
John Colvin f5ecbe8c80 refactor: Load IDataStreamFactory(s) with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-12-05 15:54:01 +00:00
John Colvin a9d55392c2 dev qol: Mark ExceptionAssert 'name' param w/ CallerArgumentExpression 2025-12-05 15:53:32 +00:00
John Colvin 96e7c5102c Add api server to docker compose 2025-12-05 15:53:32 +00:00
John Colvin 26853d597a Don't log "initializing spark config" if it has already been initialized 2025-12-05 15:53:32 +00:00
John Colvin 87d26c076b refactor: Use dependency injection in spark api server 2025-12-05 15:53:32 +00:00
John Colvin f98208a8a6 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2025-12-05 15:53:32 +00:00
John Colvin 69b1ac243e fix: Ensure spark config init and login are only performed once
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.

It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2025-12-05 15:53:31 +00:00
John Colvin 12c21f12fe refactor: Change Configuration.initialize to be async
Tweak SparkCoreServices to synchronize Configuration.init globally
2025-11-30 16:44:31 +00:00
John Colvin f0b89ffa23 refactor: Add Spark service modules for Dependency Injection 2025-11-30 16:44:18 +00:00
John Colvin a76cbafdbd cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* Compression related files
  - BZip2InputStream  (disable nullability warnings)
  - BZip2OutputStream (disable nullability warnings)
  - DeflaterHuffman   (disable nullability warnings)
  - Inflater          (disable nullability warnings)
  - InflaterDynHeader (disable nullability warnings)
  - ParallelJoinTest  (disable nullability warnings)
  - TarHeader
  - TarInputStream    (disable nullability warnings)
  - ZipEntry
  - ZipFile
  - ZipInputStream

* AsyncTCPConnection_APM
* AsyncTCPConnection_APM
* InflaterHuffmanTree
* ManagedSubscriber
* ManagedTCPMessageHub
* MapStream
* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code
* PubSubMessageReactor
* PubSubServerManagedTCPConnection
* PubSubSingleTagStateEvent
* PubSubState
* PublisherCore
* PubsubDeserializationController
* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)
* SharedStreamConfig
* SingleTagData
* SubscriberCore
* SuperMapStream
* SystemHeartbeatPublisher
* SystemStatsPublisher
* ThreadSafeQueue
* TradeBarsStream

fix build (warn): Only use Registry on windows
2025-11-21 14:54:58 -05:00
John Colvin a442e27d3d cleanup: Modernize PubSubRoute code, json parsing 2025-11-21 14:20:49 -05:00
John Colvin c2559603a8 chore: Add various collection classes
These will be useful in stream operators, row indexing, etc:
  * AggregateList
  * CircularBuffer
  * DeltaList
  * DictionarySetWrapper
  * ImmutableOrderedDictionary
  * ImmutableOrderedSet
  * IMutableCollectionFacade
    - inherits IReadOnlyCollection<T> and ICollection<T>
    - throws NotSupportedException in default impls for mutation ops
  * IMutableListFacade
    - inherits IReadOnlyList<T> and IList<T>
    - throws NotSupportedException in default impls for mutation ops
  * IndexDependentSelectorList
  * MergedSortedList
  * OffsetList
  * OrderedDictionary
  * ReverseArrayList
  * SliceList
  * StepList

More notes:
  * Renames ExceptionAssert.is_valid ExceptionAssert.Index.IsValid.
    - Like the 'ExceptionAssert.Argument' class, which provides methods to throw
      ArgumentException if a condition isn't satisfied, this class throws
      IndexOutOfRange exceptions.
2025-11-21 13:56:36 -05:00
John Colvin 8ca354e5d7 chore: Support mutations in AggregateDictionary (optionally) 2025-11-21 13:56:36 -05:00
John Colvin 10bb90c731 fix: Use IDictionary.GetEnumerator explicitly to iterate keyvals
We really need to replace these old non-generic collections with generic ones.
BlueRow's IDictionaryEnumerator implementation iterates over values when passed
to a foreach loop, not key value pairs.
2025-11-19 23:55:35 -05:00
John Colvin 2014931653 feat: Add live bbo stream 2025-11-19 10:33:31 -05:00
John Colvin 12c6b367f4 fix (preemptive): Avoid StackOverflow in md client securities fn
One of the 'securities_incremental' fns was calling back into itself instead of
the other overload. This is a 'preemptive' fix because the problematic code path
wasn't running in practice (though it was a ticking time bomb).
2025-11-19 00:36:56 -05:00
John Colvin 770e14752a chore: Add json facilities for DataStream websocket api 2025-11-19 00:36:56 -05:00
arham 91d6f80b45 streams ws defaults to nl 2025-09-16 15:33:58 +00:00
arham 77e542efa9 Fix: websocket code 2025-09-02 17:17:43 +00:00
John Colvin ff65acba05 fix: Gracefully handle missing column settings in pnl win
Pnl window settings are stored with the workspace on the server (like all shell
window settings), and may originate from an older version of Spark with a
different set of columns. We shouldn't crash if this happens.
2025-08-07 14:33:55 -04:00
John Colvin dc08945719 fix: Check proto HasPrice/QuantityDisplay before accessing props 2025-08-07 14:02:10 -04:00
John Colvin a739ca9a75 fix build: Disable warning in cs Timestamp struct
It is initialized from data stored in shared memory (not a normal struct). The
cs build emits a warning because it detects a field that "is never written to
and will always have a default value of 0."

This code hasn't changed for years. It suddenly becoming a warning-as-error is
likely related to a recent Visual Studio update.
2025-08-07 13:39:21 -04:00
John Colvin ed7b187876 feat: Add ExportToCsv button to fills win 2025-08-07 13:01:06 -04:00
John Colvin 53e1e92a6a fix: Keep existing items in SettingsPersistenceModel filtered sub
Bugs (fixed):
  * SettingsPersistenceModel drops items after initial update in filtered
    subscriptions.

    Cause: The SettingsPersistenceModel subscribe fn only publishes the "added"
    items in each update, and omits pre-existing items. From the observer's
    perspective, pre-existing items appear deleted.
2025-08-05 16:23:02 -04:00
John Colvin eee7af494a fix (fills ui): Preserve sort order during incremental update
Binary search is used to add, remove, and change rows in response to
incremental updates. Needless to say, if the rows become unsorted,
everything breaks.

Bugs (fixed):
  * The fills window crashes sometimes, possibly related to client
    reconnecting to server.

    Cause: This was a 2 part bug.
      1) Changes to existing rows are applied BEFORE the new sort index
         is determined via binary search. If the change affects a
         property in the current sorting criteria, the rows could
         become unsorted and thus invalidate the binary search.

      2) When an existing row is changed so that its new sort index is
         greater than its current one, the new index is not decremented
         by one (it must be since the row is removed before it is
         re-inserted).
2025-08-05 13:47:06 -04:00
John Colvin 9dc7611c0a fix build: Forgot to add NullableWrapper.cs a few commits back
When I made the change to format min order qty/notional cols with sec
ticks
2025-07-21 17:07:52 -04:00
John Colvin 782cdd7430 ui (orderbook): Use dark theme font color in Algo State grid
It was barely legible to against the dark background (must have been
overlooked when we originally implemented dark theme).
2025-07-21 10:02:38 -04:00
John Colvin 4184278d94 fix: Fix typo in weighted_average fn
Was multiplying distance from x by wrong weight

Bugs (fixed):
  * Pnl window avg cost is wrong when there are both start and day
    positions
2025-07-18 09:54:17 -04:00
John Colvin bfc7615c9a ui: Add "Is Aggressor" column to fills grid 2025-07-15 14:48:44 -04:00
John Colvin 6b0030559b fix build: Fill in transaction fee prop getter
TODO: Actually implement maker/taker fee. For now, just return 0.
2025-07-15 10:13:53 -04:00
Taha 0858eaac23 Polishing BlueOrb streams 2025-07-13 21:21:10 -04:00
John Colvin cc55fe1acf ui: Format min order qty/notional cols with sec ticks 2025-07-07 12:00:39 -04:00
John Colvin bb64760c62 fix (mview): Add switch case for Venue column comparer
Bugs (fixed):
  * In mview, venue column sorting is broken
2025-07-07 12:00:39 -04:00
John Colvin 9035b95612 fix: Propagate newly deleted synths in MDDaemon
There was a typo when determining "deletes" and "undeletes" on synth
changes (i.e., when deleted changes false -> true, and true-> false
respectively).
2025-06-24 10:48:42 -04:00