feat: support Flight auto-create table metadata and hint - #110
Merged
Conversation
…bulk write Attach GreptimeDB field metadata to the Arrow schema sent on the bulk (Flight) write path so the server can auto-create missing tables: - timestamp column: greptime:time_index=true, marked non-nullable - tag/field columns: greptime:semantic_type=tag|field - JSON columns: greptime:type=Json This matches the server-side contract introduced in GreptimeTeam/greptimedb-enterprise#845. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the bulk (Arrow Flight) write path to embed TableSchema semantic types into Arrow Field metadata, enabling GreptimeDB to auto-create missing tables correctly during bulk inserts (including time index handling and JSON column typing).
Changes:
- Encode column semantic types into Arrow field metadata (
greptime:time_index,greptime:semantic_type,greptime:type) when building the ArrowSchema. - Make the time index (timestamp semantic type) field non-nullable in the Arrow schema.
- Add a unit test validating the emitted metadata and nullability behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
ingester-protocol/src/main/java/io/greptime/models/ArrowHelper.java |
Adds public metadata key constants and writes semantic-type/data-type metadata into Arrow FieldType during schema construction. |
ingester-protocol/src/test/java/io/greptime/models/ArrowHelperTest.java |
Adds a unit test asserting semantic metadata and nullability for tag/timestamp/field/JSON columns. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Unify on a single metadata key: every column now carries greptime:semantic_type with value tag, field or timestamp, instead of marking the time index with a separate greptime:time_index=true key. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
v0y4g3r
changed the title
fengjiachun
approved these changes
Aug 8, 2026
fengjiachun
left a comment
fengjiachun
left a comment
Contributor
There was a problem hiding this comment.
LGTM. A few small comments inline, none blocking.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Align the Java bulk (Arrow Flight) write path with the current contract in GreptimeTeam/greptimedb-enterprise#845 (server head
9b277b5b):greptime:semantic_typewith valuetag,field, ortimestamp.timestampfield is non-nullable and identifies the unique time index.greptime:type=Json, preserving the extended type during automatic table creation.Context.withAutoCreateTable(boolean)provides a typed request-levelauto_create_tablehint. It is encoded through the existingx-greptime-hintsheader path used by Flight writes.The request hint follows server semantics: the server global auto-create option remains the upper bound; an explicit
falsedisables automatic creation for that request. Existing-table schema alignment remains available regardless of this hint.The row-based write path already sends semantic types through proto
ColumnSchema.semantic_typeand is unchanged.Verification
mvn testmvn spotless:checkgit diff --checkRelated