Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request migrates the database access layer from legacy pymysql-based raw SQL helpers to SQLAlchemy, refactoring to_sql.py to use table reflection and updating various scripts to use SQLAlchemy sessions. The code review identified several critical issues with this migration: a potential accidental full-table update in update_table_2 if columns_where is empty, runtime crashes in add_to_pages_users_db.py due to passing flat lists of scalars to session.execute(text(...)) with %s placeholders, and a crash in fixcat.py because SQLAlchemy's text() does not automatically expand list parameters in IN clauses without expanding=True. Additionally, the reviewer recommended defaulting missing values to None instead of "" to avoid type conversion errors on non-string columns, and passing dictionaries directly to .values() instead of unpacking them.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| columns_to_set = columns_to_set or [] | ||
| columns_where = columns_where or [] |
There was a problem hiding this comment.
If columns_where is empty or evaluates to False, where_clause will be empty, causing the generated UPDATE statement to execute without a WHERE clause. This will accidentally update all rows in the table. Adding a guard clause prevents catastrophic full-table updates.
| columns_to_set = columns_to_set or [] | |
| columns_where = columns_where or [] | |
| columns_to_set = columns_to_set or [] | |
| if not columns_where: | |
| raise ValueError("columns_where must be provided and non-empty to prevent accidental full-table updates.") |
| with get_session() as session: | ||
| session.execute(text(insert_qua), values) | ||
| session.commit() | ||
|
|
There was a problem hiding this comment.
Passing a flat list of scalars (values) to session.execute() with text() will crash at runtime with an ArgumentError because SQLAlchemy interprets lists of parameters as multiple parameter sets (for executemany). Additionally, text() does not natively support positional %s placeholders. Since the raw SQL query uses %s placeholders, using exec_driver_sql on the connection allows sending the query and positional parameters directly to the underlying PyMySQL driver without rewriting the query.
| with get_session() as session: | |
| session.execute(text(insert_qua), values) | |
| session.commit() | |
| with get_session() as session: | |
| session.connection().exec_driver_sql(insert_qua, tuple(values)) |
| with get_session() as session: | ||
| session.execute(text(update_qua), values) | ||
| session.commit() | ||
|
|
There was a problem hiding this comment.
Passing a flat list of scalars (values) to session.execute() with text() will crash at runtime with an ArgumentError because SQLAlchemy interprets lists of parameters as multiple parameter sets (for executemany). Additionally, text() does not natively support positional %s placeholders. Since the raw SQL query uses %s placeholders, using exec_driver_sql on the connection allows sending the query and positional parameters directly to the underlying PyMySQL driver without rewriting the query.
| with get_session() as session: | |
| session.execute(text(update_qua), values) | |
| session.commit() | |
| with get_session() as session: | |
| session.connection().exec_driver_sql(update_qua, tuple(values)) |
| with get_session() as session: | ||
| session.execute( | ||
| text("UPDATE pages SET cat = :cat WHERE title IN :titles"), | ||
| {"cat": cat, "titles": list(values)}, | ||
| ) | ||
| session.commit() |
There was a problem hiding this comment.
SQLAlchemy's text() construct does not automatically expand list/tuple parameters for IN clauses. Passing a list directly to :titles will cause a runtime crash or DBAPI error. To support list expansion, you must explicitly configure the parameter as expanding=True using bindparam.
| with get_session() as session: | |
| session.execute( | |
| text("UPDATE pages SET cat = :cat WHERE title IN :titles"), | |
| {"cat": cat, "titles": list(values)}, | |
| ) | |
| session.commit() | |
| from sqlalchemy import bindparam | |
| with get_session() as session: | |
| stmt = text("UPDATE pages SET cat = :cat WHERE title IN :titles").bindparams( | |
| bindparam("titles", expanding=True) | |
| ) | |
| session.execute(stmt, {"cat": cat, "titles": list(values)}) |
| # --- | ||
| logger.info(f"to_sql.py insert_dict({table_name}) {done} done, from {len(list_of_lines)} | batch: {lento}.") | ||
| batch = list_of_lines[i : i + lento] | ||
| values_list = [{col: row.get(col, "") for col in columns} for row in batch] |
There was a problem hiding this comment.
Using row.get(col, "") defaults missing values to an empty string "". For non-string columns (such as integers, floats, or datetimes), this can cause type conversion errors or insert incorrect default values in MySQL. Using row.get(col) (which defaults to None / NULL) is much safer and database-idiomatic.
| values_list = [{col: row.get(col, "") for col in columns} for row in batch] | |
| values_list = [{col: row.get(col) for col in columns} for row in batch] |
| for row in batch: | ||
| where_clause = [tbl.c[col] == row.get(col, "") for col in columns_where] | ||
| values = {col: row.get(col, "") for col in columns_to_set} | ||
| stmt = update(tbl).where(*where_clause).values(**values) # type: ignore[arg-type] |
There was a problem hiding this comment.
Unpacking the values dictionary as keyword arguments via **values can fail if any column name is not a valid Python identifier (e.g., contains spaces, hyphens, or is a reserved keyword). Passing the dictionary directly to .values(values) is safer and standard in SQLAlchemy.
| stmt = update(tbl).where(*where_clause).values(**values) # type: ignore[arg-type] | |
| stmt = update(tbl).where(*where_clause).values(values) |
No description provided.