John Colvin
8f3fa18bca
refactor: Rename spark_api csproj Bts.Spark.Api
...
This follows the conventional naming practices, where the project shares its
name with the namespace. Doing this in anticipation of the csharp sdk we are
about to release (for Ophir).
2026-01-04 21:08:13 +00:00
John Colvin
3565f78be8
cleanup (build): Remove redundant csproj refs
...
For example, projects that reference spark_api don't need an explicit reference
to spark_core, since spark_api already references it. The same goes for projects
that reference DataStream and DataStreamCore.
2026-01-04 21:08:13 +00:00
John Colvin
856cd552eb
refactor: Add Api Server service modules for Dependency Injection
...
* Part of a broader paradigm shift towards Dependency Injection and
Factory Pattern.
* Adds "api-server" service to docker compose
* Replaces MarketDataClient w/ IMarketDataClient in a few streams
* Replaces ViewStreamDatabase w/ IWorkScheduler in a few streams
2026-01-04 21:08:12 +00:00
John Colvin
b29b2a0adb
fix: Fix a few column types in in PubSubServerTagStatsStream
...
* total_bytes (int -> long)
* last_update_time (DateTime -> long)
2026-01-04 21:08:11 +00:00
John Colvin
76fa35382e
refactor: Add DataStream service modules for Dependency Injection
...
Namely, in DataStreamPublisher and PubSubServer. This is part of a broader
paradigm shift towards Dependency Injection and Factory Pattern.
https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection-guidelines
* Use interfaces to decouple services:
- Decouple ManagedSubscriber from ISubscriberCore impl
- Decouple PubTableBroker from IPublisherCore impl
- Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
* Model config profiles after dotnet appsettings
- Rename Configurations/<name>.json -> appsettings.<name>.json
- Make json fields match corresponding csharp props verbatim
* The Dependency Injection framework doesn't support JsonPropertyName
attributes. Config class props must match json fields verbatim.
* Move reusable startup code into new Bts.Extensions.Hosting proj
* Add pubsub and md-publisher services to compose.yaml
2026-01-04 21:08:11 +00:00
John Colvin
7d6e4efa4f
cleanup: Fix numerous build warnings in DataStreamSuite
...
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.
* Compression related files
- BZip2InputStream (disable nullability warnings)
- BZip2OutputStream (disable nullability warnings)
- DeflaterHuffman (disable nullability warnings)
- Inflater (disable nullability warnings)
- InflaterDynHeader (disable nullability warnings)
- ParallelJoinTest (disable nullability warnings)
- TarHeader
- TarInputStream (disable nullability warnings)
- ZipEntry
- ZipFile
- ZipInputStream
* AsyncTCPConnection_APM
* AsyncTCPConnection_APM
* InflaterHuffmanTree
* ManagedSubscriber
* ManagedTCPMessageHub
* MapStream
* PubSubClientMessage
- Normalize indentation/whitespace
- Fix some typos
- Address nullability issues to fix build warnings
- Modernize code
* PubSubMessageReactor
* PubSubServerManagedTCPConnection
* PubSubSingleTagStateEvent
* PubSubState
* PublisherCore
* PubsubDeserializationController
* RemoteMessage
- make 'message' nullable to resolve warnings elsewhere (in calling code)
* SharedStreamConfig
* SingleTagData
* SubscriberCore
* SuperMapStream
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SystemSubscriptionMapPublisher
* ThreadSafeQueue
* TradeBarsStream
Only use Registry on windows.
2025-12-30 11:38:20 -05:00
John Colvin
02f566a970
cleanup: Modernize PubSubRoute code, json parsing
2025-12-30 11:38:20 -05:00
John Colvin
a0b2a8b186
chore: Add async SQLiteEnumerator to read table as IBlueRowCollection
2025-12-30 11:36:35 -05:00
arham
4237d58f76
Add: metadata to dsl callback
2025-12-03 13:34:37 +00:00
arham
14a79b6352
Add:: cross exchange arb stream
2025-12-02 15:21:06 -05:00
John Colvin
2014931653
feat: Add live bbo stream
2025-11-19 10:33:31 -05:00
Taha Afzal
66c0baa150
Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln
2025-09-24 13:16:28 +00:00
arham
5305b32c04
Fix file path didnt work in linux
2025-09-17 04:28:25 +00:00
arham
e6304609f8
changes to make streams run on linux
2025-09-16 15:38:21 +00:00
arham
21d84d811d
simple changes to make stream work with wbesite
2025-09-02 17:17:43 +00:00
Taha
6b28fc4779
Lot of BlueOrb stream changes
2025-09-02 17:17:43 +00:00
Taha
0858eaac23
Polishing BlueOrb streams
2025-07-13 21:21:10 -04:00
Taha
70fae06a30
BlueOrb streams and MLP
2025-06-08 21:52:38 -04:00
Taha
a130e47218
Arb Scoring stream
2025-05-20 00:24:17 -04:00
Taha
0a6733180d
Added SpreadStatsStream
2025-05-18 01:10:33 -04:00
Taha
f2169c6bd4
Adding more data to SpreadPerformanceStream
2025-05-16 01:42:32 -04:00
Taha
23ef478391
Spread analytics streams
2025-05-15 15:01:27 -04:00
John Colvin
a6e8853cb9
fix build (warns): Account for nullable ref types
2025-04-28 17:12:58 +00:00
John Colvin
043efab5a2
build: Upgrade c# to net8.0
2025-04-28 17:12:58 +00:00
Taha
058b131760
Cleaning up some streams,new prompt etc
2025-04-04 15:36:52 -04:00
Taha
d13ba2968a
DateTimeExpanstionStream col name changes
2025-01-20 15:34:11 -05:00
Taha
461cbad338
Adding FakeBBO stream
2024-11-27 13:36:14 -05:00
Taha
aab8e83543
Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives
2024-11-26 22:33:26 -05:00
Taha
6163887cd3
Bunch of CME queries are now working
2024-10-30 13:36:23 -04:00
Taha
f9aa9dd982
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-10-15 23:48:34 -04:00
Taha
22f1ab44f9
Basic set of streams that can talks ot the LLM and get DSL back from few-shot prompt
2024-10-15 23:38:49 -04:00
Taha
288b7712db
Added and modified more streams to address teh CME NL queries
2024-10-09 13:59:01 -04:00
Taha
31df1a3827
Added benzinga streamd to MDPub's test config
2024-07-17 14:32:57 -04:00
Taha
7e219aec7c
Merging
2024-07-17 02:06:17 -04:00
Taha
37f8b98df9
Icomplete SharedStreamsConfig was breaking build, fixing
2024-07-17 02:03:43 -04:00
Taha
f06beff22f
Added new benzinga streams
2024-07-17 02:02:01 -04:00
Taha Afzal
233e2576af
Merging
2024-07-05 21:21:27 +00:00
Taha
050e21e22b
Adding TableMetaDataStream
2024-07-04 23:39:04 -04:00
Taha Afzal
672ad6d237
Cleaning Config 1.0 (not completely done)
2024-06-30 23:17:20 -04:00
Taha
0a8f6e866f
Added pubsub routes to the Title of DSQA
2024-06-27 14:04:07 -04:00
Taha
393f304be4
some buf gixes + cleanup
2024-06-26 00:23:19 -04:00
Taha
87b6a2044c
PubSubServer's linux issues fixed
2024-06-23 21:13:37 -04:00
Taha
fac0a9c25b
RemoteQueryStream
2024-06-06 09:20:05 -05:00
John Colvin
6240d89a08
Map PubSubServerConfig json to historic PubSub3Config
2024-06-03 13:46:51 +00:00
John Colvin
9050318ceb
refactor: Replace HighPerformanceTimer w/ Stopwatch
...
This is needed to build DataStreamSuite on Linux.
'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
Taha Afzal
7104e90fc9
Adding tests for stream serialization
2024-05-25 15:37:33 +00:00
Taha Afzal
07274f897a
Pubsub stats
2024-05-25 15:37:33 +00:00
Taha Afzal
03e80dabcb
CHanging member names from old style
2024-05-25 15:37:33 +00:00
Taha Afzal
54184d0875
Cleaning up ISubscriberCore to use ne PubSubClientMssage class
2024-05-25 15:37:33 +00:00
Taha Afzal
8c1342230b
RowDiff added
2024-02-08 14:11:10 -05:00