Commit Graph
12238 Commits
Author SHA1 Message Date
John Colvin 18e7658445 Add ColumnIndex class to resolve column names to indices
Improves performance compared to accessing columns by name
2024-11-26 18:53:33 +00:00
John Colvin ca522b7a23 fix Add BlueColumnTable for pandas integration 2024-11-26 18:53:33 +00:00
John Colvin 32fccbfb69 fix Add some nifty DataPacket/TableUpdate IObservable utils 2024-11-26 18:53:33 +00:00
John Colvin 27720a3127 Increase SQLiteEnumerator page size to 65536
This performs better, and doesn't seem to be too large in practice
2024-11-26 18:53:33 +00:00
John Colvin 6e3d1c7b8e Serialize TimeSpan w/ microsecond precision in DataPacket json 2024-11-26 18:53:33 +00:00
John Colvin 625738b7b5 Make 'name' default to CallerArgumentExpression for convenience 2024-11-26 18:53:33 +00:00
John Colvin e906bfec64 fix rename snake_case PascalCase 2024-11-26 18:53:33 +00:00
John Colvin 6b2845aa90 Add GetValue fn to Variant, PrimitiveVariant 2024-11-26 18:53:33 +00:00
John Colvin 65fbea141d Add BlueColumnTable for pandas integration
This will make it easier/more natural to support pandas column expressions
2024-11-26 18:53:33 +00:00
John Colvin c6635005a1 refactor: Rename snake_case fns PascalCase 2024-11-26 18:53:33 +00:00
John Colvin 72c21912a1 fix generate c# code for row types 2024-11-26 18:53:33 +00:00
John Colvin 3142e62a10 fix generate more row types with t4 2024-11-26 18:53:33 +00:00
John Colvin 1a82f5152e fix load IDataStreamFactory impls with dependency injection 2024-11-26 18:53:33 +00:00
John Colvin 722ab02d2b Format DateTimes w/ microsecond precision in DataPacketJsonConverter 2024-11-26 18:53:33 +00:00
John Colvin 6f352a5c95 fix Add some nifty DataPacket/TableUpdate IObservable 2024-11-26 18:53:33 +00:00
John Colvin ef3f7cc430 Generate some more row types with new t4 templates 2024-11-26 18:53:33 +00:00
John Colvin 17b98adf74 fix load idatastreamfactory impls 2024-11-26 18:53:33 +00:00
John Colvin 809734eca8 fix: Import all IDataStreamFactory instances in publisher service 2024-11-26 18:53:33 +00:00
John Colvin ec87a2311d fix: Add Ipv6 urls to Kestrel config
The vite dev proxy server was routing requests to ipv6
loopback, ::1, instead of ipv4, 127.0.0.1
2024-11-26 18:53:33 +00:00
John Colvin 58bdb4e00d cleanup: Add key attributes to jsx code
They were causing warnings in console log
2024-11-26 18:53:32 +00:00
John Colvin ee629278cf cleanup: Add semi-colons to js code 2024-11-26 18:53:32 +00:00
John Colvin 09ca8f504f Add logs for vite dev proxy server 2024-11-26 18:53:32 +00:00
John Colvin fd6926eee8 Incorporate env files into docker compose
In order to support local (ephemeral) config. For example, to run sparkd
with replay md to troubleshoot an issue.
2024-11-26 18:53:32 +00:00
John Colvin 5ae46fef77 fix: Run MDReplayer on background thread (don't block)
This was preventing other services from starting up
2024-11-26 18:53:32 +00:00
John Colvin 07d12cbf94 Add BlueRowColumnMask to hide columns w/ minimal overhead
Less overhead than ProjectStream
2024-11-26 18:53:32 +00:00
John Colvin e75dd34213 fix build: Update protobuf namespaces in c# 2024-11-26 18:53:32 +00:00
John Colvin 73294f1b76 fix build: Fix win cpp build after lots of datastream work on linux
* Fix protoc path
* Disable tests (the catch2 build from vcpkg uses incorrect RuntimeIdentifier).
  TODO: Need to look into this.
* Update cmake target name from 'libspark' -> 'spark' in vs_filters.cmake
* Disable concurrencpp (async coroutines) in HttpClient
* Update paths in GetWindowsClientSources.cmake
* Update libs in GetWindowsClientLinkLibraries.cmake
* Handful of other trivial changes
2024-11-26 18:53:32 +00:00
John Colvin 1ca34db259 fix build: Move SecurityTests.cpp into "tests" dir
We build outside of the main lib (libspark in this case), and instead
build them in a separate exe.
2024-11-26 18:53:32 +00:00
John Colvin 7a3804f1cd Update datastream webapp appsettings.json (config) 2024-11-26 18:53:32 +00:00
John Colvin f03b571f05 Generate a few row types with new t4 templates 2024-11-26 18:53:32 +00:00
John Colvin 07aa53dffa fix SQLiteEnumerator 2024-11-26 18:53:32 +00:00
John Colvin 68b42bdb01 Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection 2024-11-26 18:53:32 +00:00
John Colvin 2d7e6b1ec2 build: Generate c# code for RowTypes
Produces structs that implement IBlueRow with strongly typed
properties for each column.
2024-11-26 18:53:32 +00:00
John Colvin db0df50b53 Add some nifty DataPacket/TableUpdate IObservable utils 2024-11-26 18:53:32 +00:00
John Colvin bb6662a2c1 fix: Remove query from literal error stream
The parser can't handle colons in the expression (it throws/crashes)
2024-11-26 18:53:32 +00:00
John Colvin e1f98a3209 Tweak new streams database config 2024-11-26 18:53:32 +00:00
John Colvin 1f3b2c5cab fix build: Disable some includes on windows 2024-11-26 18:53:32 +00:00
John Colvin cd1f4631dd build: Move lib_spark tests into separate exe
So we don't have to run them through sparkd
2024-11-26 18:53:32 +00:00
John Colvin 4a1a7308dd refactor: Move catch_test_runner from spark (lib) to sparkd (app)
This source file provides the entry point for catch2
2024-11-26 18:53:32 +00:00
John Colvin 7da668c136 Add gen purpose blue row/table utils for data streams
* Add BlueRowReflection<T> so ViewStream implementations can use
  strong row types without needing to implement IBlueRow
* Add default implementations for IDictionary members in IBlueRow to make
  implementing IBlueRow less tedious
2024-11-26 18:53:32 +00:00
John Colvin 304e669de2 Extend dsl to support explicit param names 2024-11-26 18:53:32 +00:00
John Colvin 9d86c85686 Remove erroneous "DSL: " prefix in ai prompt stream
This causes parse errors since we extended the dsl syntax to support named
params in the form of "name: arg".
2024-11-26 18:53:32 +00:00
John Colvin 410680cadf Configure directory browser endpt for vite dev server 2024-11-26 18:53:32 +00:00
John Colvin 64bd5eef4f Load IDataStremFactory impls with dependency injection
So we can create ViewStreams that depend on other services, config, etc.
Previously, the PrimitiveManager relied purely on static construction of types
through reflection.
2024-11-26 18:53:32 +00:00
John Colvin 3619d4b22c refactor: Use generic collections in PrimitiveManager
The class that instantiates ViewStreams via reflection
2024-11-26 18:53:32 +00:00
John Colvin baa9dbfe76 Fix typo in NLStream's namespace 2024-11-26 18:53:32 +00:00
John Colvin 11241b9b37 Fix LiteralStream syntax in md publisher appsettings.json 2024-11-26 18:53:32 +00:00
John Colvin eb3fad64b0 fix build: Upgrade Newtonsoft.Json 13.0.2 -> 13.0.3 in spark_websocket
The DataStreamSuite.sln build was broken because other projects in the sln were
already using the newer version (can't mix and match).
2024-11-26 18:53:32 +00:00
John Colvin bdfb00aa22 Add c# Variant class, modeled after std::variant in c++ 2024-11-26 18:53:32 +00:00
John Colvin 50611fe4a3 Add generic ViewStream, BlueRow types/interfaces 2024-11-26 18:53:32 +00:00