While working on #872, I found that passing raw numpy scalar values (e.g. np.int64(5), not run through pandas or .item()) directly into insert_all() doesn't get converted to a normal SQLite INTEGER/REAL — instead it silently stores garbage binary data as a BLOB. Repro:
python
import sqlite_utils, numpy as np
db = sqlite_utils.Database(memory=True)
db["t"].insert_all([{"col": np.int64(5)}])
print(list(db["t"].rows))
{'col': b'\x05\x00\x00\x00\x00\x00\x00\x00'} instead of {'col': 5}
This likely affects anyone using numpy arrays directly without pandas. Might be worth adding numpy-scalar handling to jsonify_if_needed() in db.py.
While working on #872, I found that passing raw numpy scalar values (e.g. np.int64(5), not run through pandas or .item()) directly into insert_all() doesn't get converted to a normal SQLite INTEGER/REAL — instead it silently stores garbage binary data as a BLOB. Repro:
python
import sqlite_utils, numpy as np
db = sqlite_utils.Database(memory=True)
db["t"].insert_all([{"col": np.int64(5)}])
print(list(db["t"].rows))
{'col': b'\x05\x00\x00\x00\x00\x00\x00\x00'} instead of {'col': 5}
This likely affects anyone using numpy arrays directly without pandas. Might be worth adding numpy-scalar handling to jsonify_if_needed() in db.py.