Commit Graph
163 Commits
Author SHA1 Message Date
Taha fddd310a03 Added new benzinga streams 2024-10-03 17:55:30 +05:00
Taha d2a2d5fb37 Adding TableMetaDataStream 2024-10-03 17:55:30 +05:00
Taha Afzal 4b76442a8f Cleaning Config 1.0 (not completely done) 2024-10-03 17:55:30 +05:00
Taha 09b863f2ab Added pubsub routes to the Title of DSQA 2024-10-03 17:55:30 +05:00
Taha dd221aba10 Fixing the bug where messages were being sent by the pubsubserver out of order 2024-10-03 17:55:28 +05:00
Taha 5ef862b026 some buf gixes + cleanup 2024-10-03 17:54:28 +05:00
Taha cc43e9d3a4 PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop 2024-10-03 17:52:47 +05:00
Taha b6eb727bc9 PubSubServer's linux issues fixed 2024-10-03 17:52:38 +05:00
Taha 5cf7eacd82 Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe 2024-06-24 00:28:07 -04:00
John Colvin 5620f33524 vscode: Add task to view logs for last PubSubServer run 2024-06-08 14:42:13 +00:00
John Colvin 3b631e0d5f Fix a couple more typos in logs 2024-06-07 16:35:06 +00:00
John Colvin 23d7d3775d cleanup: Delete unused files 2024-06-07 10:40:34 -05:00
John Colvin 4940771166 fix: Don't login to sparkd from DSQA 2024-06-07 10:02:42 -05:00
John Colvin 8c797ee76b Fix typo in log msg 2024-06-06 09:21:01 -05:00
John Colvin ce80957d07 DSQA shouldn't connect to sparkd by default 2024-06-06 09:21:01 -05:00
John Colvin c60018b59e DataStreamPublisher shouldn't connect to sparkd by default
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin 3a27af87d1 PubSubServer shouldn't download config from sparkd by default
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin 068c1c6973 cleanup: Remove erroneous comment 2024-06-06 09:21:01 -05:00
John Colvin 4a2735839e Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin d2e9fe8237 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin cfc60766d8 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin b03fec7f79 Change line endings to crlf in a few cs files 2024-06-06 09:20:59 -05:00
John Colvin e44601d1b1 fix merge: Bring back few fns that SchemaGuesserStream needs
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha 07bade53c0 Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB 2024-06-06 09:20:05 -05:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 45f02391d1 cleanup: Remove sparkd login creds from PubSubServerConfig
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin a3ca06addc Reduce 'internal_publish' log level to Diag
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin 32869e54ec fix: Fix typos in c# Logs
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin ca73220a2e Add new PubSub cfg to DSQA
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin 9fc3c2618e cleanup: Remove unnecessary reference to SharedStreams
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin e72e0c0a30 fix: Make PubSubServer actually use sparkd url in config 2024-06-06 09:15:17 -05:00
John Colvin a5bfc75278 cleanup: Fix a few nullability warnings in DSQA 2024-06-06 09:15:17 -05:00
John Colvin 261a184d25 refactor: Replace legacy Crono type w/ dotnet Stopwatch 2024-06-06 09:15:16 -05:00
John Colvin 17e60df32b cleanup: Remove unused file PubSub3Config.cs 2024-06-06 09:15:16 -05:00
John Colvin c99d5fe24c Add support for core PubSub config via json 2024-06-06 09:15:16 -05:00
John Colvin 8a3e4edb4d Dispose connections in SimpleTCPServer.Dispose 2024-06-06 09:15:15 -05:00
John Colvin c425145fc9 Remove mongo stream from MarketDataPublisher.json
It doesn't work unless you have mongodb running (a bit of a burden when you
just want to publish market data)
2024-06-06 09:15:15 -05:00
John Colvin 4855935ae0 improvement: Get PubSubRoutes from config (json)
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin 65f412ec9f fix: Fix myLoadAllTypes impl
* Fix lowercase typo in hardcoded assemblies (case matters now that we
  run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
  and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin ab2d36e523 vscode: Add tasks.json
This provides a good base of operations to build/launch our now numerous
projects
2024-06-06 09:15:09 -05:00
John Colvin 5e7e7e8ef4 test: Add FilterStreamTests 2024-06-05 15:44:35 +00:00
John Colvin 549708fcc2 fix: Use ".dll" file ext when loading assemblies by path 2024-06-05 15:44:35 +00:00
John Colvin 015d846941 feat: Add Pandas(Script)Stream 2024-06-05 15:44:35 +00:00
John Colvin 6240d89a08 Map PubSubServerConfig json to historic PubSub3Config 2024-06-03 13:46:51 +00:00
John Colvin c76700e5e5 cleanup: Remove unused PubSubRoute array from DSQA 2024-06-03 13:46:51 +00:00
John Colvin a8a0e8dea0 cleanup: Remove unused PubSubConfigureForm 2024-06-03 13:46:51 +00:00
John Colvin d65a2d6ed6 test: Add DataPacketBuilderTests
This tests somes facilities used in the new MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin 1824d5f9a4 feat: Add [incremental] MongoCollectionStream 2024-06-03 13:46:51 +00:00
John Colvin d3eb29087b refactor: Impl IDictionary<int, IBlueRow> in BlueTable 2024-06-03 13:46:51 +00:00
John Colvin d6fae238f9 Add IObservable ViewStream wrapper 2024-06-03 13:46:51 +00:00