Skip to content

perf(bigquery): avoid duplicate FieldList deserialization in Schema.fromPb and FieldList.fromPb - #15003

Open
lqiu96 wants to merge 1 commit into
googleapis:mainfrom
lqiu96:perf/bq-t1-14-fieldlist-exponential-frompb
Open

lqiu96 wants to merge 1 commit into
googleapis:mainfrom
lqiu96:perf/bq-t1-14-fieldlist-exponential-frompb

Conversation

@lqiu96

@lqiu96 lqiu96 commented Nov 13, 2026

Copy link
Copy Markdown
Member

Summary

Avoids duplicate Field and FieldList deserialization when parsing TableSchema and TableFieldSchema responses (Schema.fromPb, FieldList.fromPb, and Field.fromPb).

Previously, two compounding inefficiencies occurred on every schema parse:

  1. Double evaluation of non-caching Lists.transform views ($2^D$ work on nested schemas): FieldList.fromPb and Field.fromPb wrapped TableFieldSchema lists in Guava's Lists.transform(..., Field.FROM_PB_FUNCTION), which does not cache converted elements. Inside FieldList(Iterable<Field>), ImmutableList.copyOf(fields) already forced all elements to be allocated immediately on Pass 1, and for (Field field : fields) iterated the lazy view a second time just to populate nameIndex, discarding the duplicate Field and FieldList allocations. For nested RECORD / STRUCT schemas of depth $D$, Field.fromPb executed $2^D$ times instead of once.
  2. Duplicate FieldList construction in Schema.fromPb: Schema.fromPb called Schema.of(FieldList.fromPb(fields)), which passed an already-constructed FieldList into FieldList.of(Iterable<Field>), copying it into a second FieldList and rebuilding nameIndex.

Changes

  • FieldList:
    • Iterate the already-materialized this.fields list in FieldList(Iterable<Field>) instead of re-iterating the input Iterable.
    • Short-circuit FieldList.of(Iterable<Field>) when the input is already a FieldList.
    • Materialize FieldList.fromPb(List<TableFieldSchema>) eagerly using ImmutableList.builderWithExpectedSize.
  • Field:
    • Call FieldList.fromPb directly in Field.fromPb for nested subfields instead of wrapping in Lists.transform.
  • FieldListTest:
    • Added testSingleEvaluationAndReuseInOf verifying single evaluation of lazy iterables and instance reuse in FieldList.of and Schema.of.

Verification

  • mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=FieldListTest (5 passed)
  • mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=FieldTest (12 passed)
  • mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=SchemaTest (3 passed)
  • mvn test -pl java-bigquery/google-cloud-bigquery -Dtest=FieldValueListTest (5 passed)

@lqiu96
lqiu96 requested review from a team as code owners November 13, 2026 20:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the creation and deserialization of FieldList objects in the BigQuery client. It replaces lazy Lists.transform views with eager materialization when converting fields from Protocol Buffers (fromPb), avoiding duplicate evaluations and deserialization of nested subfields. Additionally, it optimizes FieldList.of to directly return the input if it is already an immutable FieldList instance, and updates the FieldList constructor to iterate over its materialized fields rather than the input iterable. A new unit test has been added to verify these optimizations. I have no feedback to provide as there are no review comments.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant