petefisher78/spark
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13,318 Commits 163 Branches 302 Tags
fd0fb1f76015416b64f3158d09e455172394de57
Commit Graph
13318 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taha Afzal fd0fb1f760 Adding incremental book updates to IMarketDataListener 2024-08-12 22:49:27 -04:00
Taha 5cf7eacd82 Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe 2024-06-24 00:28:07 -04:00
John Colvin 087d8ab8c9 Set text=auto in gitattributes 
Otherwise git can cause files to show as "permanently modified" when you
apply changes from the index (e.g., by running 'git rebase', or 'git
stash pop')
 2024-06-11 17:39:12 +00:00
John Colvin b69b7b3350 vscode: Use '\' dir separator in show_last_run.cmd 
This script only runs on Windows
 2024-06-08 15:08:25 -05:00
John Colvin 5620f33524 vscode: Add task to view logs for last PubSubServer run 2024-06-08 14:42:13 +00:00
John Colvin f3e33fa390 vscode: Add task to run PubSubServer without building 2024-06-07 16:41:42 +00:00
John Colvin 3b631e0d5f Fix a couple more typos in logs 2024-06-07 16:35:06 +00:00
John Colvin f93d28ee76 vscode: Add task to view logs for last DSQA run 2024-06-07 11:34:36 -05:00
John Colvin 23d7d3775d cleanup: Delete unused files 2024-06-07 10:40:34 -05:00
John Colvin 4940771166 fix: Don't login to sparkd from DSQA 2024-06-07 10:02:42 -05:00
John Colvin c1967a8f6a vscode: Add tasks to edit Spark, DataStream config 2024-06-06 11:12:56 -05:00
John Colvin 8c797ee76b Fix typo in log msg 2024-06-06 09:21:01 -05:00
John Colvin ce80957d07 DSQA shouldn't connect to sparkd by default 2024-06-06 09:21:01 -05:00
John Colvin c60018b59e DataStreamPublisher shouldn't connect to sparkd by default 
It only needs to connect to sparkd if it's publishing market data
 2024-06-06 09:21:01 -05:00
John Colvin 3a27af87d1 PubSubServer shouldn't download config from sparkd by default 
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
 2024-06-06 09:21:01 -05:00
John Colvin 068c1c6973 cleanup: Remove erroneous comment 2024-06-06 09:21:01 -05:00
John Colvin 645a58c97d vscode: Add launch.json 2024-06-06 09:21:01 -05:00
John Colvin b7f455b29a vscode: Remove launch/tasks.json from gitignore 2024-06-06 09:21:00 -05:00
John Colvin 4a2735839e Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin d2e9fe8237 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin cfc60766d8 Change line endings to crlf in a few cs files 2024-06-06 09:21:00 -05:00
John Colvin b03fec7f79 Change line endings to crlf in a few cs files 2024-06-06 09:20:59 -05:00
John Colvin fb8e7c5b81 Ensure crlf line endings for cs files 2024-06-06 09:20:05 -05:00
John Colvin e44601d1b1 fix merge: Bring back few fns that SchemaGuesserStream needs 
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
 2024-06-06 09:20:05 -05:00
Taha 07bade53c0 Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB 2024-06-06 09:20:05 -05:00
Taha fac0a9c25b RemoteQueryStream 2024-06-06 09:20:05 -05:00
John Colvin 1d1de3dd18 Ensure proper line endings for sh, cmd scripts 
Via new gitattributes file
 2024-06-06 09:20:05 -05:00
John Colvin cfcbf004ba vscode: Use consistent naming conventions in tasks.json 2024-06-06 09:20:05 -05:00
John Colvin 78c7dadbbe vscode: Add parameterless task variants 
This allows us to build as a 'preLaunchTask' when debugging (without
requiring users to redundantly specify 'Debug' as the build type).
 2024-06-06 09:20:01 -05:00
John Colvin 7b5a6031af build: Use consistent naming conventions in CMake presets 
We'll use similar naming conventions for our (vscode) tasks
 2024-06-06 09:15:17 -05:00
John Colvin c13a7782c8 build: Name sparkd binary dirs after cmake preset 
All other builds follow this pattern. Given the fact that we have so
many different builds now (spark, sparkd, data-streams, api-server,
etc.), it behooves us to adopt consistent naming conventions.
 2024-06-06 09:15:17 -05:00
John Colvin 45f02391d1 cleanup: Remove sparkd login creds from PubSubServerConfig 
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
 2024-06-06 09:15:17 -05:00
John Colvin a3ca06addc Reduce 'internal_publish' log level to Diag 
It was a bit noisy for the Info level
 2024-06-06 09:15:17 -05:00
John Colvin 32869e54ec fix: Fix typos in c# Logs 
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
 2024-06-06 09:15:17 -05:00
John Colvin ca73220a2e Add new PubSub cfg to DSQA 
At least the server address (port) needs to be exposed
 2024-06-06 09:15:17 -05:00
John Colvin 9fc3c2618e cleanup: Remove unnecessary reference to SharedStreams 
From DataStreamPublisher
 2024-06-06 09:15:17 -05:00
John Colvin e72e0c0a30 fix: Make PubSubServer actually use sparkd url in config 2024-06-06 09:15:17 -05:00
John Colvin a5bfc75278 cleanup: Fix a few nullability warnings in DSQA 2024-06-06 09:15:17 -05:00
John Colvin 261a184d25 refactor: Replace legacy Crono type w/ dotnet Stopwatch 2024-06-06 09:15:16 -05:00
John Colvin 54e96a06cb build: Build DSQA on Windows in build_data_stream.py 
It was only building DataStreamSuite.sln before, which is crossplatform
now so it no longer includes DSQA.
 2024-06-06 09:15:16 -05:00
John Colvin 17e60df32b cleanup: Remove unused file PubSub3Config.cs 2024-06-06 09:15:16 -05:00
John Colvin c99d5fe24c Add support for core PubSub config via json 2024-06-06 09:15:16 -05:00
John Colvin 6049826e90 build: Make build_data_stream.py cross platform 
Previously it only worked on Windows
 2024-06-06 09:15:16 -05:00
John Colvin 672149dd89 Add more descriptive logs (via stdout) during initialization 
The C++ configure/initialize routines were particularly opaque
 2024-06-06 09:15:16 -05:00
John Colvin 8c99147f04 build: Build cmake presets in distinct binary dirs 
It may use more disk space, but we really don't have a choice. CMake
projects with different parameters, preprocessors, source files, etc.
are not designed to share binary dirs. For example, the libspark.so file
that the Spark front-end is supposed to use is not necessarily the same
as the libspark.so file the Api Server is supposed use.
 2024-06-06 09:15:16 -05:00
John Colvin 4d4b0b7ae3 build: Give api server distinct shm prefix 2024-06-06 09:15:16 -05:00
John Colvin 1a7955c1b1 build: Remove unnecessary cache var/macro for api server 
* BUILD_API_SERVER (cmake cache var)
* SPARK_ENABLE_API_SERVER (preprocessor defined conditionally w/ BUILD_API_SERVER)

They were originally added to control whether or not MD Consumer extern
fns were built. They have been made redundant by the more appropriately
named EXPORT_MD_CONSUMER cache var/macro (both have same name now).
 2024-06-06 09:15:16 -05:00
John Colvin f8fc2a1d96 cleanup: Remove static service Locator class 
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
 2024-06-06 09:15:16 -05:00
John Colvin 9a7b3cb77c refactor: Reduce code (simplify) in BTS.init 
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
 2024-06-06 09:15:16 -05:00
John Colvin 8a3e4edb4d Dispose connections in SimpleTCPServer.Dispose 2024-06-06 09:15:15 -05:00