Taha
6604eaaea7
some buf gixes + cleanup
2024-09-18 14:02:34 +00:00
Taha
29e5c72969
PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop
2024-09-18 14:02:34 +00:00
Taha
21e07e1b2e
PubSubServer's linux issues fixed
2024-09-18 14:02:34 +00:00
Taha
5cf7eacd82
Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe
2024-06-24 00:28:07 -04:00
John Colvin
5620f33524
vscode: Add task to view logs for last PubSubServer run
2024-06-08 14:42:13 +00:00
John Colvin
3b631e0d5f
Fix a couple more typos in logs
2024-06-07 16:35:06 +00:00
John Colvin
23d7d3775d
cleanup: Delete unused files
2024-06-07 10:40:34 -05:00
John Colvin
4940771166
fix: Don't login to sparkd from DSQA
2024-06-07 10:02:42 -05:00
John Colvin
8c797ee76b
Fix typo in log msg
2024-06-06 09:21:01 -05:00
John Colvin
ce80957d07
DSQA shouldn't connect to sparkd by default
2024-06-06 09:21:01 -05:00
John Colvin
c60018b59e
DataStreamPublisher shouldn't connect to sparkd by default
...
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin
3a27af87d1
PubSubServer shouldn't download config from sparkd by default
...
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin
068c1c6973
cleanup: Remove erroneous comment
2024-06-06 09:21:01 -05:00
John Colvin
4a2735839e
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
d2e9fe8237
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
cfc60766d8
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
b03fec7f79
Change line endings to crlf in a few cs files
2024-06-06 09:20:59 -05:00
John Colvin
e44601d1b1
fix merge: Bring back few fns that SchemaGuesserStream needs
...
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha
07bade53c0
Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB
2024-06-06 09:20:05 -05:00
Taha
fac0a9c25b
RemoteQueryStream
2024-06-06 09:20:05 -05:00
John Colvin
45f02391d1
cleanup: Remove sparkd login creds from PubSubServerConfig
...
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin
a3ca06addc
Reduce 'internal_publish' log level to Diag
...
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin
32869e54ec
fix: Fix typos in c# Logs
...
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin
ca73220a2e
Add new PubSub cfg to DSQA
...
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin
9fc3c2618e
cleanup: Remove unnecessary reference to SharedStreams
...
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin
e72e0c0a30
fix: Make PubSubServer actually use sparkd url in config
2024-06-06 09:15:17 -05:00
John Colvin
a5bfc75278
cleanup: Fix a few nullability warnings in DSQA
2024-06-06 09:15:17 -05:00
John Colvin
261a184d25
refactor: Replace legacy Crono type w/ dotnet Stopwatch
2024-06-06 09:15:16 -05:00
John Colvin
17e60df32b
cleanup: Remove unused file PubSub3Config.cs
2024-06-06 09:15:16 -05:00
John Colvin
c99d5fe24c
Add support for core PubSub config via json
2024-06-06 09:15:16 -05:00
John Colvin
8a3e4edb4d
Dispose connections in SimpleTCPServer.Dispose
2024-06-06 09:15:15 -05:00
John Colvin
c425145fc9
Remove mongo stream from MarketDataPublisher.json
...
It doesn't work unless you have mongodb running (a bit of a burden when you
just want to publish market data)
2024-06-06 09:15:15 -05:00
John Colvin
4855935ae0
improvement: Get PubSubRoutes from config (json)
...
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin
65f412ec9f
fix: Fix myLoadAllTypes impl
...
* Fix lowercase typo in hardcoded assemblies (case matters now that we
run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin
ab2d36e523
vscode: Add tasks.json
...
This provides a good base of operations to build/launch our now numerous
projects
2024-06-06 09:15:09 -05:00
John Colvin
5e7e7e8ef4
test: Add FilterStreamTests
2024-06-05 15:44:35 +00:00
John Colvin
549708fcc2
fix: Use ".dll" file ext when loading assemblies by path
2024-06-05 15:44:35 +00:00
John Colvin
015d846941
feat: Add Pandas(Script)Stream
2024-06-05 15:44:35 +00:00
John Colvin
6240d89a08
Map PubSubServerConfig json to historic PubSub3Config
2024-06-03 13:46:51 +00:00
John Colvin
c76700e5e5
cleanup: Remove unused PubSubRoute array from DSQA
2024-06-03 13:46:51 +00:00
John Colvin
a8a0e8dea0
cleanup: Remove unused PubSubConfigureForm
2024-06-03 13:46:51 +00:00
John Colvin
d65a2d6ed6
test: Add DataPacketBuilderTests
...
This tests somes facilities used in the new MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin
1824d5f9a4
feat: Add [incremental] MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin
d3eb29087b
refactor: Impl IDictionary<int, IBlueRow> in BlueTable
2024-06-03 13:46:51 +00:00
John Colvin
d6fae238f9
Add IObservable ViewStream wrapper
2024-06-03 13:46:51 +00:00
John Colvin
5741ec9ab2
cleanup: Remove unnecessary usings (namespaces)
2024-06-03 13:46:51 +00:00
John Colvin
596975b309
style: Apply modern formatting to Crono
2024-06-03 13:46:51 +00:00
John Colvin
9050318ceb
refactor: Replace HighPerformanceTimer w/ Stopwatch
...
This is needed to build DataStreamSuite on Linux.
'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
John Colvin
7ee6933789
refactor: Add formal config classes for prepublish streams
...
They can be passed to command line via --config <json_file>
2024-06-03 13:46:51 +00:00
John Colvin
306c572b53
improvement: Polish clis in DataStreamSuite
...
Specifically PubSubServer, DataStreamPublisher and DSQA
* Use CommandLineParser (nuget pkg)
* Clean up csproj files a bit while we're at it
2024-06-03 13:46:51 +00:00