Commit Graph
64 Commits
Author SHA1 Message Date
John Colvin 3ce9f57e95 fix: Make sure transient streams are attached to IServiceProvider
So they can retrieve services necessary to function (e.g., add themselves
to the master IWorkScheduler so doWork gets called, etc.)
2024-07-07 18:14:01 -04:00
John Colvin 5fa284cad3 refactor: Use DependencyInjection paradigm in DataStreamSuite
This allows us to create loosely coupled services with automatic
dependency resolution
2024-07-07 18:13:59 -04:00
John Colvin 8acf07d23c cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-07-07 17:47:39 -04:00
John Colvin 0bc3394724 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-07-07 17:45:35 -04:00
Taha Afzal 4fb3a15ef0 Added pss_echo() 2024-07-07 10:24:09 -04:00
Taha bcfc89f95b Adding TableMetaDataStream 2024-07-07 10:24:09 -04:00
Taha Afzal 0026ecbfaa Cleaning Config 1.0 (not completely done) 2024-07-07 10:24:09 -04:00
Taha 5dc67084eb some buf gixes + cleanup 2024-07-07 10:23:41 -04:00
Taha 46bc00ce87 PubSubServer's linux issues fixed 2024-07-07 10:23:12 -04:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d6fae238f9 Add IObservable ViewStream wrapper 2024-06-03 13:46:51 +00:00
John Colvin 5741ec9ab2 cleanup: Remove unnecessary usings (namespaces) 2024-06-03 13:46:51 +00:00
John Colvin 9050318ceb refactor: Replace HighPerformanceTimer w/ Stopwatch
This is needed to build DataStreamSuite on Linux.

'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
John Colvin e95997ea17 cleanup: Chip away a few nullability warnings in DataStreamSuite
This is a start, but there're still > 700 of them!
2024-05-25 15:37:33 +00:00
Taha Afzal 90ec64d25c deleting old serializers 2024-05-25 15:37:33 +00:00
Taha Afzal 061481419e Streams can be published and subscribed now 2024-05-25 15:37:33 +00:00
Taha Afzal 8035255868 Cleaned up the cashe held by PublisherCore 2024-05-25 15:37:33 +00:00
Taha Afzal 7feaf869dc Stream serialization revamp, all tests pass (TestsStream.s) 2024-05-25 15:37:33 +00:00
Taha Afzal 7104e90fc9 Adding tests for stream serialization 2024-05-25 15:37:33 +00:00
Taha Afzal 07274f897a Pubsub stats 2024-05-25 15:37:33 +00:00
Taha Afzal f626e53415 CPublisherCore, SubscriberCore, PubsubMessageReactor claned 2024-05-25 15:37:33 +00:00
Taha Afzal 888c06262b PublisherCore cleaned up 2024-05-25 15:37:33 +00:00
Taha Afzal 90bcf5a455 Adding testss for PubSubMessages 2024-05-25 15:37:33 +00:00
Taha Afzal 4510e120ab enabling some streams like max, that were dependent on AddCOlumns (thta John fixed in the last commit) 2024-02-12 13:21:28 -05:00
Taha Afzal da2af06ee6 Merging 2024-02-08 15:48:51 -05:00
John Colvin 8660dea85a build: Downgrade new DataStream code from net8 -> net7
The AddColumnsStream, SortStream, and requisite yacc/lex grammar were
originally added to a new branch, off of development, which is on net8.
I forgot to change it to net7 when I cherry-picked the commits for the
regular development branch.

It's wasn't quite as simple as flipping net8 -> net7 in csproj files,
because I was eager to use new language features only available in net8!
Namely, primary constructors and collection expressions.
2024-02-08 14:24:31 -06:00
Taha Afzal 8c1342230b RowDiff added 2024-02-08 14:11:10 -05:00
John Colvin d0ddc1d57b Resurrect SortStream
Add ColumnSortExpressions grammar (lex/yacc)
2024-02-08 10:04:53 -06:00
John Colvin fe1e953030 Resurrect AddColumnStream
Add ColumnExpressions grammar (lex/yacc)
2024-02-08 10:04:53 -06:00
Taha Afzal 8dd2775092 Added volatility to trade bars 2024-02-04 23:22:52 -05:00
Taha Afzal 287ee291cc Adding a stream that send a TCP packet to the echo server and monitors the n_sends and n_recvs 2024-01-25 13:37:41 -05:00
Taha Afzal cc3e289345 KeyValueToRelationalStream added 2024-01-16 19:08:40 -05:00
Taha Afzal 8cbbbc6da3 Merging 2024-01-13 23:45:51 -05:00
Taha Afzal 047267ad1b AsyncTCPConnection_APM completed (not hooked yet) 2024-01-06 11:34:46 -05:00
Taha Afzal 5b4b41d4eb Adding MongoDB astream integration 2024-01-02 00:44:52 -05:00
Taha Afzal a7189628b1 Adding EMA, SMA and RenkoStreams 2023-12-18 12:42:14 -05:00
Taha Afzal 2ae3ded65e src/DataStreamSuite/DataStream/Primitives/FileStreams/PlaybackStream.cs 2023-11-14 18:15:53 +00:00
Taha Afzal 4acb1e268a Merging 2023-11-14 18:14:18 +00:00
John Colvin dcac260efe fix (hack): Pass lines as array to cloud stream
Rather than a single multi-line string. This hack will have to do
until we implement multi-line strings in the parser.
2023-10-25 17:42:59 -04:00
Taha 88bf04e5f1 TimeSeries and TimeBucketStreams, Stream offloading etc 2023-10-23 14:18:31 -04:00
Taha 33030b1212 Market data streams work now 2023-10-16 00:01:29 -04:00
Taha fa4da63c24 Streams can be published across PubSub now 2023-09-30 12:59:02 -04:00
John Colvin 9311c27f47 feat: Resurrect DataStream primitives
They were originally commented out because the expression parsing utils
weren't ready (now they are).
2023-09-29 13:10:50 -04:00
John Colvin b416209ca7 refactor: Add DataStream cfg protos
Translated from Edge config
2023-09-29 13:10:49 -04:00
John Colvin ed0be9162b refactor: Move PrimitiveManager to DataStreamCore
So DSQA can reference it
2023-09-29 13:10:49 -04:00
John Colvin 248e37577f feat: Add DataStreamExpressions
All lex/yacc scanners/parsers are now generated with a proper dotnet
nuget pkg, YaccLexTools.

* Implements BooleanExpressionFilter with the new Parser.
2023-09-29 13:10:49 -04:00
John Colvin ac1ede5986 refactor: Move a few low-level DataStream types to DataStreamCore
So the lex/yacc generators can reference them
2023-09-29 13:10:49 -04:00
John Colvin d1f5ae86fd chore: Get rid of BooleanExpressionsInterpreter csproj 2023-09-29 13:10:49 -04:00