Skip to content

Raw numpy scalar values get silently stored as corrupted BLOBs instead of being converted #876

Description

@aayushgupta7725

While working on #872, I found that passing raw numpy scalar values (e.g. np.int64(5), not run through pandas or .item()) directly into insert_all() doesn't get converted to a normal SQLite INTEGER/REAL — instead it silently stores garbage binary data as a BLOB. Repro:

python
import sqlite_utils, numpy as np
db = sqlite_utils.Database(memory=True)
db["t"].insert_all([{"col": np.int64(5)}])
print(list(db["t"].rows))

{'col': b'\x05\x00\x00\x00\x00\x00\x00\x00'} instead of {'col': 5}

This likely affects anyone using numpy arrays directly without pandas. Might be worth adding numpy-scalar handling to jsonify_if_needed() in db.py.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions