Taha
f06beff22f
Added new benzinga streams
2024-07-17 02:02:01 -04:00
Taha
050e21e22b
Adding TableMetaDataStream
2024-07-04 23:39:04 -04:00
Taha Afzal
672ad6d237
Cleaning Config 1.0 (not completely done)
2024-06-30 23:17:20 -04:00
Taha
0a8f6e866f
Added pubsub routes to the Title of DSQA
2024-06-27 14:04:07 -04:00
Taha
b177ffef04
Fixing the bug where messages were being sent by the pubsubserver out of order
2024-06-27 00:17:22 -04:00
Taha
393f304be4
some buf gixes + cleanup
2024-06-26 00:23:19 -04:00
Taha
151816bf5e
PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop
2024-06-24 02:29:28 -04:00
Taha
87b6a2044c
PubSubServer's linux issues fixed
2024-06-23 21:13:37 -04:00
John Colvin
23d7d3775d
cleanup: Delete unused files
2024-06-07 10:40:34 -05:00
John Colvin
4940771166
fix: Don't login to sparkd from DSQA
2024-06-07 10:02:42 -05:00
John Colvin
8c797ee76b
Fix typo in log msg
2024-06-06 09:21:01 -05:00
John Colvin
ce80957d07
DSQA shouldn't connect to sparkd by default
2024-06-06 09:21:01 -05:00
John Colvin
c60018b59e
DataStreamPublisher shouldn't connect to sparkd by default
...
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin
3a27af87d1
PubSubServer shouldn't download config from sparkd by default
...
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin
068c1c6973
cleanup: Remove erroneous comment
2024-06-06 09:21:01 -05:00
John Colvin
4a2735839e
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
d2e9fe8237
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
cfc60766d8
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
b03fec7f79
Change line endings to crlf in a few cs files
2024-06-06 09:20:59 -05:00
John Colvin
e44601d1b1
fix merge: Bring back few fns that SchemaGuesserStream needs
...
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha
07bade53c0
Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB
2024-06-06 09:20:05 -05:00
Taha
fac0a9c25b
RemoteQueryStream
2024-06-06 09:20:05 -05:00
John Colvin
45f02391d1
cleanup: Remove sparkd login creds from PubSubServerConfig
...
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin
a3ca06addc
Reduce 'internal_publish' log level to Diag
...
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin
32869e54ec
fix: Fix typos in c# Logs
...
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin
ca73220a2e
Add new PubSub cfg to DSQA
...
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin
9fc3c2618e
cleanup: Remove unnecessary reference to SharedStreams
...
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin
e72e0c0a30
fix: Make PubSubServer actually use sparkd url in config
2024-06-06 09:15:17 -05:00
John Colvin
a5bfc75278
cleanup: Fix a few nullability warnings in DSQA
2024-06-06 09:15:17 -05:00
John Colvin
261a184d25
refactor: Replace legacy Crono type w/ dotnet Stopwatch
2024-06-06 09:15:16 -05:00
John Colvin
17e60df32b
cleanup: Remove unused file PubSub3Config.cs
2024-06-06 09:15:16 -05:00
John Colvin
c99d5fe24c
Add support for core PubSub config via json
2024-06-06 09:15:16 -05:00
John Colvin
672149dd89
Add more descriptive logs (via stdout) during initialization
...
The C++ configure/initialize routines were particularly opaque
2024-06-06 09:15:16 -05:00
John Colvin
f8fc2a1d96
cleanup: Remove static service Locator class
...
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
2024-06-06 09:15:16 -05:00
John Colvin
9a7b3cb77c
refactor: Reduce code (simplify) in BTS.init
...
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
2024-06-06 09:15:16 -05:00
John Colvin
8a3e4edb4d
Dispose connections in SimpleTCPServer.Dispose
2024-06-06 09:15:15 -05:00
John Colvin
e59183bb1e
improvement: Allow app data dir to be overriden w/ env var
...
Historically, this has been hardcoded to ~/sparkdata on Linux, and
%APPDATA%/bts on Windows.
2024-06-06 09:15:15 -05:00
John Colvin
8d22e91c0d
fix build: Remove references to HighPerformanceTimer
...
An old custom timer class from Edge that we no longer need. We're
replacing it w/ a dotnet builtin timer class called 'Stopwatch'.
2024-06-06 09:15:15 -05:00
John Colvin
c425145fc9
Remove mongo stream from MarketDataPublisher.json
...
It doesn't work unless you have mongodb running (a bit of a burden when you
just want to publish market data)
2024-06-06 09:15:15 -05:00
John Colvin
4855935ae0
improvement: Get PubSubRoutes from config (json)
...
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin
65f412ec9f
fix: Fix myLoadAllTypes impl
...
* Fix lowercase typo in hardcoded assemblies (case matters now that we
run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin
40681c6093
Add MaybeNullWhen(false) attribute to TryGetFirst result
...
So callers can assume non-null when it returns true (for IEnumerables of
non-null ref types)
2024-06-06 09:15:15 -05:00
John Colvin
ab2d36e523
vscode: Add tasks.json
...
This provides a good base of operations to build/launch our now numerous
projects
2024-06-06 09:15:09 -05:00
John Colvin
5e7e7e8ef4
test: Add FilterStreamTests
2024-06-05 15:44:35 +00:00
John Colvin
9291d30ae9
build: Add cmake opt to export MDConsumerInterop fns
...
Now that we have all these different builds (sparkd, spark, api server,
data streams) - many of which are cross platform - we need a proper
toggle for this (relying on 'IF WIN32' preprocessors doesn't cut it)
2024-06-05 15:44:35 +00:00
John Colvin
549708fcc2
fix: Use ".dll" file ext when loading assemblies by path
2024-06-05 15:44:35 +00:00
John Colvin
fc48cc64af
fix build (webapp): Add --include=dev arg to npm install
...
The apt repo finally updated npm from 8.X.X to 10.8.0. This broke the
Spark.Admin.WebApp build because the new version doesn't install dev
dependencies by default.
2024-06-05 15:44:35 +00:00
John Colvin
015d846941
feat: Add Pandas(Script)Stream
2024-06-05 15:44:35 +00:00
John Colvin
e39108bbda
fix build (sparkd): Fix typo
...
'git' was appended to a type name. I suspect a keyboard focus conflict
between vs code editor/terminal panes.
2024-06-03 13:46:51 +00:00
John Colvin
6240d89a08
Map PubSubServerConfig json to historic PubSub3Config
2024-06-03 13:46:51 +00:00