Taha
f06beff22f
Added new benzinga streams
2024-07-17 02:02:01 -04:00
Taha
050e21e22b
Adding TableMetaDataStream
2024-07-04 23:39:04 -04:00
Taha Afzal
672ad6d237
Cleaning Config 1.0 (not completely done)
2024-06-30 23:17:20 -04:00
Taha
0a8f6e866f
Added pubsub routes to the Title of DSQA
2024-06-27 14:04:07 -04:00
Taha
b177ffef04
Fixing the bug where messages were being sent by the pubsubserver out of order
2024-06-27 00:17:22 -04:00
Taha
393f304be4
some buf gixes + cleanup
2024-06-26 00:23:19 -04:00
Taha
151816bf5e
PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop
2024-06-24 02:29:28 -04:00
Taha
87b6a2044c
PubSubServer's linux issues fixed
2024-06-23 21:13:37 -04:00
John Colvin
f93d28ee76
vscode: Add task to view logs for last DSQA run
2024-06-07 11:34:36 -05:00
John Colvin
23d7d3775d
cleanup: Delete unused files
2024-06-07 10:40:34 -05:00
John Colvin
4940771166
fix: Don't login to sparkd from DSQA
2024-06-07 10:02:42 -05:00
John Colvin
c1967a8f6a
vscode: Add tasks to edit Spark, DataStream config
2024-06-06 11:12:56 -05:00
John Colvin
8c797ee76b
Fix typo in log msg
2024-06-06 09:21:01 -05:00
John Colvin
ce80957d07
DSQA shouldn't connect to sparkd by default
2024-06-06 09:21:01 -05:00
John Colvin
c60018b59e
DataStreamPublisher shouldn't connect to sparkd by default
...
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin
3a27af87d1
PubSubServer shouldn't download config from sparkd by default
...
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin
068c1c6973
cleanup: Remove erroneous comment
2024-06-06 09:21:01 -05:00
John Colvin
645a58c97d
vscode: Add launch.json
2024-06-06 09:21:01 -05:00
John Colvin
b7f455b29a
vscode: Remove launch/tasks.json from gitignore
2024-06-06 09:21:00 -05:00
John Colvin
4a2735839e
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
d2e9fe8237
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
cfc60766d8
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
b03fec7f79
Change line endings to crlf in a few cs files
2024-06-06 09:20:59 -05:00
John Colvin
fb8e7c5b81
Ensure crlf line endings for cs files
2024-06-06 09:20:05 -05:00
John Colvin
e44601d1b1
fix merge: Bring back few fns that SchemaGuesserStream needs
...
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha
07bade53c0
Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB
2024-06-06 09:20:05 -05:00
Taha
fac0a9c25b
RemoteQueryStream
2024-06-06 09:20:05 -05:00
John Colvin
1d1de3dd18
Ensure proper line endings for sh, cmd scripts
...
Via new gitattributes file
2024-06-06 09:20:05 -05:00
John Colvin
cfcbf004ba
vscode: Use consistent naming conventions in tasks.json
2024-06-06 09:20:05 -05:00
John Colvin
78c7dadbbe
vscode: Add parameterless task variants
...
This allows us to build as a 'preLaunchTask' when debugging (without
requiring users to redundantly specify 'Debug' as the build type).
2024-06-06 09:20:01 -05:00
John Colvin
7b5a6031af
build: Use consistent naming conventions in CMake presets
...
We'll use similar naming conventions for our (vscode) tasks
2024-06-06 09:15:17 -05:00
John Colvin
c13a7782c8
build: Name sparkd binary dirs after cmake preset
...
All other builds follow this pattern. Given the fact that we have so
many different builds now (spark, sparkd, data-streams, api-server,
etc.), it behooves us to adopt consistent naming conventions.
2024-06-06 09:15:17 -05:00
John Colvin
45f02391d1
cleanup: Remove sparkd login creds from PubSubServerConfig
...
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin
a3ca06addc
Reduce 'internal_publish' log level to Diag
...
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin
32869e54ec
fix: Fix typos in c# Logs
...
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin
ca73220a2e
Add new PubSub cfg to DSQA
...
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin
9fc3c2618e
cleanup: Remove unnecessary reference to SharedStreams
...
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin
e72e0c0a30
fix: Make PubSubServer actually use sparkd url in config
2024-06-06 09:15:17 -05:00
John Colvin
a5bfc75278
cleanup: Fix a few nullability warnings in DSQA
2024-06-06 09:15:17 -05:00
John Colvin
261a184d25
refactor: Replace legacy Crono type w/ dotnet Stopwatch
2024-06-06 09:15:16 -05:00
John Colvin
54e96a06cb
build: Build DSQA on Windows in build_data_stream.py
...
It was only building DataStreamSuite.sln before, which is crossplatform
now so it no longer includes DSQA.
2024-06-06 09:15:16 -05:00
John Colvin
17e60df32b
cleanup: Remove unused file PubSub3Config.cs
2024-06-06 09:15:16 -05:00
John Colvin
c99d5fe24c
Add support for core PubSub config via json
2024-06-06 09:15:16 -05:00
John Colvin
6049826e90
build: Make build_data_stream.py cross platform
...
Previously it only worked on Windows
2024-06-06 09:15:16 -05:00
John Colvin
672149dd89
Add more descriptive logs (via stdout) during initialization
...
The C++ configure/initialize routines were particularly opaque
2024-06-06 09:15:16 -05:00
John Colvin
8c99147f04
build: Build cmake presets in distinct binary dirs
...
It may use more disk space, but we really don't have a choice. CMake
projects with different parameters, preprocessors, source files, etc.
are not designed to share binary dirs. For example, the libspark.so file
that the Spark front-end is supposed to use is not necessarily the same
as the libspark.so file the Api Server is supposed use.
2024-06-06 09:15:16 -05:00
John Colvin
4d4b0b7ae3
build: Give api server distinct shm prefix
2024-06-06 09:15:16 -05:00
John Colvin
1a7955c1b1
build: Remove unnecessary cache var/macro for api server
...
* BUILD_API_SERVER (cmake cache var)
* SPARK_ENABLE_API_SERVER (preprocessor defined conditionally w/ BUILD_API_SERVER)
They were originally added to control whether or not MD Consumer extern
fns were built. They have been made redundant by the more appropriately
named EXPORT_MD_CONSUMER cache var/macro (both have same name now).
2024-06-06 09:15:16 -05:00
John Colvin
f8fc2a1d96
cleanup: Remove static service Locator class
...
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
2024-06-06 09:15:16 -05:00
John Colvin
9a7b3cb77c
refactor: Reduce code (simplify) in BTS.init
...
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
2024-06-06 09:15:16 -05:00