Commit Graph
71 Commits
Author SHA1 Message Date
John Colvin 443aac540a fix: Add null check to NonIncGroupByStream 2024-10-14 17:21:46 +00:00
John Colvin 7550e81aa7 fix: Add null check in TimeBasedFilter.cs 2024-10-14 17:21:46 +00:00
John Colvin 5085e872cb Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-10-14 17:21:45 +00:00
John Colvin c0fd30ab92 refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-10-14 17:21:45 +00:00
John Colvin 40dfbd90d5 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-10-14 17:21:45 +00:00
John Colvin 9411e8ed21 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-10-14 17:21:45 +00:00
John Colvin 55f7a86500 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-10-14 17:21:45 +00:00
John Colvin de62ac6e82 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-10-14 17:21:45 +00:00
John Colvin 3b5e8d6e7e cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-10-14 17:21:45 +00:00
Taha 322c06bfbe Fixing build relted to BCG vs bts namespace 2024-08-21 16:24:22 -04:00
Taha b68fceb362 Cleaning up FileStreams 2024-08-21 16:11:57 -04:00
Taha f06beff22f Added new benzinga streams 2024-07-17 02:02:01 -04:00
Taha 050e21e22b Adding TableMetaDataStream 2024-07-04 23:39:04 -04:00
Taha Afzal 672ad6d237 Cleaning Config 1.0 (not completely done) 2024-06-30 23:17:20 -04:00
Taha 393f304be4 some buf gixes + cleanup 2024-06-26 00:23:19 -04:00
Taha 87b6a2044c PubSubServer's linux issues fixed 2024-06-23 21:13:37 -04:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d6fae238f9 Add IObservable ViewStream wrapper 2024-06-03 13:46:51 +00:00
John Colvin 5741ec9ab2 cleanup: Remove unnecessary usings (namespaces) 2024-06-03 13:46:51 +00:00
John Colvin 9050318ceb refactor: Replace HighPerformanceTimer w/ Stopwatch
This is needed to build DataStreamSuite on Linux.

'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
John Colvin e95997ea17 cleanup: Chip away a few nullability warnings in DataStreamSuite
This is a start, but there're still > 700 of them!
2024-05-25 15:37:33 +00:00
Taha Afzal 90ec64d25c deleting old serializers 2024-05-25 15:37:33 +00:00
Taha Afzal 061481419e Streams can be published and subscribed now 2024-05-25 15:37:33 +00:00
Taha Afzal 8035255868 Cleaned up the cashe held by PublisherCore 2024-05-25 15:37:33 +00:00
Taha Afzal 7feaf869dc Stream serialization revamp, all tests pass (TestsStream.s) 2024-05-25 15:37:33 +00:00
Taha Afzal 7104e90fc9 Adding tests for stream serialization 2024-05-25 15:37:33 +00:00
Taha Afzal 07274f897a Pubsub stats 2024-05-25 15:37:33 +00:00
Taha Afzal f626e53415 CPublisherCore, SubscriberCore, PubsubMessageReactor claned 2024-05-25 15:37:33 +00:00
Taha Afzal 888c06262b PublisherCore cleaned up 2024-05-25 15:37:33 +00:00
Taha Afzal 90bcf5a455 Adding testss for PubSubMessages 2024-05-25 15:37:33 +00:00
Taha Afzal 4510e120ab enabling some streams like max, that were dependent on AddCOlumns (thta John fixed in the last commit) 2024-02-12 13:21:28 -05:00
Taha Afzal da2af06ee6 Merging 2024-02-08 15:48:51 -05:00
John Colvin 8660dea85a build: Downgrade new DataStream code from net8 -> net7
The AddColumnsStream, SortStream, and requisite yacc/lex grammar were
originally added to a new branch, off of development, which is on net8.
I forgot to change it to net7 when I cherry-picked the commits for the
regular development branch.

It's wasn't quite as simple as flipping net8 -> net7 in csproj files,
because I was eager to use new language features only available in net8!
Namely, primary constructors and collection expressions.
2024-02-08 14:24:31 -06:00
Taha Afzal 8c1342230b RowDiff added 2024-02-08 14:11:10 -05:00
John Colvin d0ddc1d57b Resurrect SortStream
Add ColumnSortExpressions grammar (lex/yacc)
2024-02-08 10:04:53 -06:00
John Colvin fe1e953030 Resurrect AddColumnStream
Add ColumnExpressions grammar (lex/yacc)
2024-02-08 10:04:53 -06:00
Taha Afzal 8dd2775092 Added volatility to trade bars 2024-02-04 23:22:52 -05:00
Taha Afzal 287ee291cc Adding a stream that send a TCP packet to the echo server and monitors the n_sends and n_recvs 2024-01-25 13:37:41 -05:00
Taha Afzal cc3e289345 KeyValueToRelationalStream added 2024-01-16 19:08:40 -05:00
Taha Afzal 8cbbbc6da3 Merging 2024-01-13 23:45:51 -05:00
Taha Afzal 047267ad1b AsyncTCPConnection_APM completed (not hooked yet) 2024-01-06 11:34:46 -05:00
Taha Afzal 5b4b41d4eb Adding MongoDB astream integration 2024-01-02 00:44:52 -05:00
Taha Afzal a7189628b1 Adding EMA, SMA and RenkoStreams 2023-12-18 12:42:14 -05:00
Taha Afzal 2ae3ded65e src/DataStreamSuite/DataStream/Primitives/FileStreams/PlaybackStream.cs 2023-11-14 18:15:53 +00:00
Taha Afzal 4acb1e268a Merging 2023-11-14 18:14:18 +00:00
John Colvin dcac260efe fix (hack): Pass lines as array to cloud stream
Rather than a single multi-line string. This hack will have to do
until we implement multi-line strings in the parser.
2023-10-25 17:42:59 -04:00
Taha 88bf04e5f1 TimeSeries and TimeBucketStreams, Stream offloading etc 2023-10-23 14:18:31 -04:00
Taha 33030b1212 Market data streams work now 2023-10-16 00:01:29 -04:00