Skip to content

fix(bigquery): Fix timestamp scientific notation handling in _helpers.py#17278

Open
pkenjora wants to merge 1 commit into
googleapis:mainfrom
pkenjora:patch-1
Open

fix(bigquery): Fix timestamp scientific notation handling in _helpers.py#17278
pkenjora wants to merge 1 commit into
googleapis:mainfrom
pkenjora:patch-1

Conversation

@pkenjora
Copy link
Copy Markdown

Casting to float first handles scientific notation returned by API. Otherwise its a noop.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #17277🦕

Casting to float first handles scientific notation returned by API.  
Otherwise its a noop.
@pkenjora pkenjora requested review from a team as code owners May 27, 2026 20:48
@pkenjora pkenjora requested review from sycai and removed request for a team May 27, 2026 20:48
@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 27, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the timestamp conversion helper in _helpers.py to cast values to float before converting them to integers. The reviewer noted that casting all values to float can cause precision loss for large integer timestamps (beyond the year 2285) and suggested attempting to parse as an integer first, falling back to float only if a ValueError occurs.

if _not_null(value, field):
# value will be a integer in seconds, to microsecond precision, in UTC.
return _datetime_from_microseconds(int(value))
return _datetime_from_microseconds(int(float(value)))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Directly casting all values to float before converting to int can lead to precision loss for very large integer values (specifically, integers greater than 2^53 - 1, which corresponds to timestamps beyond the year 2285). Since BigQuery supports timestamps up to the year 9999, we should attempt to parse the value as an integer first to preserve full precision, and only fall back to float parsing if a ValueError occurs (e.g., when scientific notation or a decimal point is present).

            try:
                microseconds = int(value)
            except ValueError:
                microseconds = int(float(value))
            return _datetime_from_microseconds(microseconds)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

@sycai sycai changed the title Fix timestamp scientific notation handling in _helpers.py fix(bigquery): Fix timestamp scientific notation handling in _helpers.py May 27, 2026
@sycai
Copy link
Copy Markdown
Contributor

sycai commented May 27, 2026

Hello @pkenjora ! Could you also provide test coverage for this change? Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Timestamp Returned In Scientific Notation By API Crashes Field Cast

2 participants