Skip to content

JVM bulk loads: the subnormal check walks every row when the text contains "e-3" (UUIDs, hex ids) #182

Closed

Description

@tae898

Since #180, the JVM bulk JSON paths (insert_many, graph batches) check each chunk for a subnormal double by searching the JSON text for "e-3" and, on a hit, walking every row in Python (type_conversion._dumps_checked). "e-3" is common in ordinary text: hex ids and UUIDs (...4fe-3a...), names like name-3. A load whose rows hold such strings takes the per-row Python walk on every chunk, for a value (a subnormal double) that almost never occurs.

Fix: search for the exponent json.dumps writes for a subnormal (e-308 to e-329 after a digit) with a compiled pattern, which costs the same C-speed scan and no longer matches ordinary text. A test checks that UUID-like and name-3 text never reaches the walk, and that a subnormal still takes the per-row path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions