feat(gizmosql): add - #289
Open
HMDank wants to merge 8 commits into
Open
Conversation
dargmuesli
reviewed
Aug 6, 2026
Comment on lines
+8
to
+13
| # Tombstone messages (Debezium deletes) carry no payload and must be dropped. | ||
| - bloblang: | | ||
| root = if this.payload.after.or(this.after) == null { deleted() } | ||
| - mapping: | | ||
| root = this.payload.after.or(this.after) | ||
| root.operation = this.payload.op.or(this.op) |
Member
There was a problem hiding this comment.
Not sure if this handles all tombstone messages correctly, I'd need to research a bit more.
dargmuesli
reviewed
Aug 16, 2026
| GF_DATABASE_PASSWORD__FILE: /run/secrets/postgres-role-service-grafana-password | ||
| GF_DATABASE_TYPE: postgres | ||
| GF_DATABASE_USER__FILE: /run/secrets/postgres-role-service-grafana-username | ||
| GF_INSTALL_PLUGINS: gizmodata-gizmosql-datasource |
Member
There was a problem hiding this comment.
I think about removing all the provisioning code for Grafana and instead configure it in the live system. Provisioning codifies the initial setup and makes it more explicit in that sense but also is quite rigid and disallows changing the config in the live Grafana instance. As we want to persist configurations done in the Grafana UI anyway, we can safe ourselves from config being too rigid and extensive and just configure those connectors in the UI, I'd say.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces an end-to-end event stream analytics pipeline, enabling Redpanda event data to be ingested, stored as Parquet files in cloud storage, and queried using Arrow Flight SQL via GizmoSQL. The changes add new services, configuration, and documentation to support this workflow.
Event Stream Analytics Pipeline
redpanda-connectservice, which ingests events from Redpanda, transforms the data, and writes Parquet files to S3-compatible cloud storage (R2). Includes configuration for S3 credentials and data schema.gizmosqlservice, which allows querying the event stream's Parquet lakehouse using Arrow Flight SQL. Includes secure credential handling and a custom entrypoint script for environment setup and templating.