Commit Graph
79 Commits
Author SHA1 Message Date
John Colvin ea0a1fbcd3 refactor: Load IDataStreamFactory(s) with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of
types through reflection.
2025-11-19 13:53:18 -05:00
John Colvin 7566b2560f refactor: Change namespace to Bts.DataStreamCore for a few types
Since they're in the DataStreamCore csproj
2025-11-19 13:29:16 -05:00
John Colvin f81dc08621 cleanup: Remove unused ViewStreamDatabase config 2025-11-19 13:23:43 -05:00
John Colvin d603adcf7a Remove reference to spark protobuf Config in ViewStream
We can't read spark protobuf Config until it has been initialized,
and since ViewStream doesn't have the configServer param handy it
isn't able to initialize config if necessary. In that case, it would simply
block until config was initialized by someone else. Such a risk doesn't
seem justified for this particular configuration dependency
(DataStreamConfig.UseUncopiedNewRowsInFire). For now, its hardcoded to false.
2025-11-19 13:23:43 -05:00
John Colvin 95683de1c6 Assert ViewStreamDatabase.instance is only created once
The instance must be initialized via ViewStreamDatabase.SetGlobalInstance
2025-11-19 13:23:42 -05:00
John Colvin 00c8081516 cleanup: Remove unused code from ViewStreamDatabase 2025-11-19 13:23:40 -05:00
John Colvin 12ea9a060a fix: Add a few null checks
* TimeBasedFilter
* TableAccumulator2
* NonIncGroupByStream
2025-11-19 13:21:51 -05:00
John Colvin 6e09854337 Add ErrorChanged event to propagate DataStream exceptions 2025-11-19 13:21:50 -05:00
John Colvin f51d0ae973 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2025-11-19 12:27:20 -05:00
John Colvin 520f6311c7 refactor: Only support 1 tag type in IPubSubServerTagHandler 2025-11-19 12:24:39 -05:00
John Colvin a19658c1e3 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2025-11-19 12:24:39 -05:00
John Colvin c66511a089 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2025-11-19 12:24:38 -05:00
John Colvin f571b4992f Add generic ViewStream, BlueRow types/interfaces 2025-11-19 12:24:37 -05:00
John Colvin 1d70eecb44 Add BlueRowColumnMask to hide columns w/ minimal overhead
Less overhead than ProjectStream
2025-11-19 12:24:36 -05:00
John Colvin 69632d7327 fix Impl IDictionary<int, IBlueRow> in BlueTable 2025-11-19 12:24:36 -05:00
John Colvin 2014931653 feat: Add live bbo stream 2025-11-19 10:33:31 -05:00
John Colvin aa740e786c refactor: Automatically add id column in BlueTableSchemaBuilder 2025-11-19 00:36:56 -05:00
John Colvin 5080960b97 refactor: Replace DataPacket w/ IDataPacket in stream handlers
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-19 00:36:56 -05:00
John Colvin 770e14752a chore: Add json facilities for DataStream websocket api 2025-11-19 00:36:56 -05:00
John Colvin e24b877e58 Add gen purpose blue row/table utils for data streams
* Add BlueRowReflection<T> so ViewStream implementations can use
  strong row types without needing to implement IBlueRow
* Add default implementations for IDictionary members in IBlueRow to make
  implementing IBlueRow less tedious
2025-11-19 00:31:53 -05:00
Taha Afzal 09abe4a432 Added RowBasedAppendStream 2025-11-18 03:10:30 +00:00
Taha Afzal 66c0baa150 Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln 2025-09-24 13:16:28 +00:00
John Colvin 7e79a21cd4 Add some nifty DataPacket/TableUpdate IObservable utils 2025-07-15 10:13:53 -04:00
Taha 0858eaac23 Polishing BlueOrb streams 2025-07-13 21:21:10 -04:00
John Colvin 5163ca4cae refactor: Organize libspark proj w/ stuttering include
This new structure provides several benefits:

* Allows us to install multiple CMake projects in the same destination
  dir (without include path conflicts).
  - This makes shipping the trading algo sdk a trivial excercise. Simply
    copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
  modules, which are easier to manage.

Windows specific notes:

* Get rid of unnecessary script vs_filters.cmake
* Define WebsocketConnectionGateway directly in cs (instead of proto)
  - The proto would no longer be in the include path passed to protoc.
    Moving it to c# because there wasn't a compelling reason for it to
    be a proto in the first place.
* Set spark output prefix to "lib" on windows
  - So we get "libspark.dll" instead of "spark.dll", which conflicts
    with the csharp lib, "Spark.dll".
* Copy libspark.dll to net8.0 & net8.0-windows in a single cmake build
  target

fix build: Copy libspark to both dirs net8.0, net8.0-windows
2025-06-16 11:08:53 -04:00
Taha 70fae06a30 BlueOrb streams and MLP 2025-06-08 21:52:38 -04:00
John Colvin 043efab5a2 build: Upgrade c# to net8.0 2025-04-28 17:12:58 +00:00
Taha aab8e83543 Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives 2024-11-26 22:33:26 -05:00
Taha 6163887cd3 Bunch of CME queries are now working 2024-10-30 13:36:23 -04:00
Taha f9aa9dd982 Merge branch 'development' of bitbucket.org:bluetradesys/spark into development 2024-10-15 23:48:34 -04:00
Taha 22f1ab44f9 Basic set of streams that can talks ot the LLM and get DSL back from few-shot prompt 2024-10-15 23:38:49 -04:00
Taha 288b7712db Added and modified more streams to address teh CME NL queries 2024-10-09 13:59:01 -04:00
Taha 7e219aec7c Merging 2024-07-17 02:06:17 -04:00
Taha f06beff22f Added new benzinga streams 2024-07-17 02:02:01 -04:00
Taha Afzal 233e2576af Merging 2024-07-05 21:21:27 +00:00
Taha 050e21e22b Adding TableMetaDataStream 2024-07-04 23:39:04 -04:00
Taha Afzal 672ad6d237 Cleaning Config 1.0 (not completely done) 2024-06-30 23:17:20 -04:00
Taha 393f304be4 some buf gixes + cleanup 2024-06-26 00:23:19 -04:00
Taha 87b6a2044c PubSubServer's linux issues fixed 2024-06-23 21:13:37 -04:00
John Colvin cfc60766d8 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin b03fec7f79 Change line endings to crlf in a few cs files 2024-06-06 09:20:59 -05:00
John Colvin e44601d1b1 fix merge: Bring back few fns that SchemaGuesserStream needs
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 65f412ec9f fix: Fix myLoadAllTypes impl
* Fix lowercase typo in hardcoded assemblies (case matters now that we
  run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
  and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin 5e7e7e8ef4 test: Add FilterStreamTests 2024-06-05 15:44:35 +00:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d3eb29087b refactor: Impl IDictionary<int, IBlueRow> in BlueTable 2024-06-03 13:46:51 +00:00
John Colvin 5741ec9ab2 cleanup: Remove unnecessary usings (namespaces) 2024-06-03 13:46:51 +00:00
John Colvin 9050318ceb refactor: Replace HighPerformanceTimer w/ Stopwatch
This is needed to build DataStreamSuite on Linux.

'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00