Commit Graph
182 Commits
Author SHA1 Message Date
John Colvin 475bf1effc cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-09-18 14:02:35 +00:00
John Colvin ec1e657687 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-09-18 14:02:35 +00:00
John Colvin 284c84dfe6 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-09-18 14:02:35 +00:00
John Colvin b78c61bf44 fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-09-18 14:02:35 +00:00
John Colvin b9ef3ddf91 Fix typo in log msg 2024-09-18 14:02:35 +00:00
John Colvin 658472b2a8 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-09-18 14:02:35 +00:00
John Colvin e5bbc26da5 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-09-18 14:02:35 +00:00
John Colvin d77ac1600b fix build (warn): Only use Registry on windows 2024-09-18 14:02:35 +00:00
John Colvin 3b41a2065a cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-09-18 14:02:35 +00:00
John Colvin f993f01634 cleanup: Normalize indentation in TradeBarsStream 2024-09-18 14:02:35 +00:00
John Colvin e7067a708b cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-09-18 14:02:35 +00:00
John Colvin 04ac2b9d3d cleanup: Modernize PubSubRoute code, json parsing 2024-09-18 14:02:35 +00:00
John Colvin 66c6b0bf8c cleanup: Remove unused DateTime grammar from exp parser 2024-09-18 14:02:34 +00:00
John Colvin e928295e71 Add json facilities for DataStream websocket api 2024-09-18 14:02:34 +00:00
John Colvin 8f7409b3e7 cleanup: Remove empty file (clearly added by accident) 2024-09-18 14:02:34 +00:00
Taha Afzal 8c06d84a8b Fixing problems where PSS's client socket's IP was different than what the client was sending in PubSubClientSubscription 2024-09-18 14:02:34 +00:00
Taha Afzal 941186a12a Fixing PubsubEchoTestStream 2024-09-18 14:02:34 +00:00
Taha Afzal 0984007c6e Added pss_echo() 2024-09-18 14:02:34 +00:00
Taha Afzal 2bb7a89777 Merging 2024-09-18 14:02:34 +00:00
Taha Afzal 15fc3cb785 src/DataStreamSuite/PubSub/ManagedTCP/AsyncTCPConnection_AsyncWrapper.cs 2024-09-18 14:02:34 +00:00
Taha 725396589e Adding TableMetaDataStream 2024-09-18 14:02:34 +00:00
Taha Afzal 6fc1ec4d67 Cleaning Config 1.0 (not completely done) 2024-09-18 14:02:34 +00:00
Taha 0496b299dd Added pubsub routes to the Title of DSQA 2024-09-18 14:02:34 +00:00
Taha b55c4103df Fixing the bug where messages were being sent by the pubsubserver out of order 2024-09-18 14:02:34 +00:00
Taha 6604eaaea7 some buf gixes + cleanup 2024-09-18 14:02:34 +00:00
Taha 29e5c72969 PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop 2024-09-18 14:02:34 +00:00
Taha 21e07e1b2e PubSubServer's linux issues fixed 2024-09-18 14:02:34 +00:00
Taha 5cf7eacd82 Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe 2024-06-24 00:28:07 -04:00
John Colvin 5620f33524 vscode: Add task to view logs for last PubSubServer run 2024-06-08 14:42:13 +00:00
John Colvin 3b631e0d5f Fix a couple more typos in logs 2024-06-07 16:35:06 +00:00
John Colvin 23d7d3775d cleanup: Delete unused files 2024-06-07 10:40:34 -05:00
John Colvin 4940771166 fix: Don't login to sparkd from DSQA 2024-06-07 10:02:42 -05:00
John Colvin 8c797ee76b Fix typo in log msg 2024-06-06 09:21:01 -05:00
John Colvin ce80957d07 DSQA shouldn't connect to sparkd by default 2024-06-06 09:21:01 -05:00
John Colvin c60018b59e DataStreamPublisher shouldn't connect to sparkd by default
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin 3a27af87d1 PubSubServer shouldn't download config from sparkd by default
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin 068c1c6973 cleanup: Remove erroneous comment 2024-06-06 09:21:01 -05:00
John Colvin 4a2735839e Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin d2e9fe8237 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin cfc60766d8 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin b03fec7f79 Change line endings to crlf in a few cs files 2024-06-06 09:20:59 -05:00
John Colvin e44601d1b1 fix merge: Bring back few fns that SchemaGuesserStream needs
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha 07bade53c0 Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB 2024-06-06 09:20:05 -05:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 45f02391d1 cleanup: Remove sparkd login creds from PubSubServerConfig
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin a3ca06addc Reduce 'internal_publish' log level to Diag
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin 32869e54ec fix: Fix typos in c# Logs
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin ca73220a2e Add new PubSub cfg to DSQA
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin 9fc3c2618e cleanup: Remove unnecessary reference to SharedStreams
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin e72e0c0a30 fix: Make PubSubServer actually use sparkd url in config 2024-06-06 09:15:17 -05:00