Skip to content

Change default encoding option used by pandas.read_csv - #150

Merged
phargogh merged 4 commits into
natcap:mainfrom
davemfish:bugfix/143-utf8-bom
Sep 4, 2026
Merged

phargogh merged 4 commits into
natcap:mainfrom
davemfish:bugfix/143-utf8-bom

Conversation

@davemfish

@davemfish davemfish commented Sep 4, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #143 .
Fixes #147

I confirmed the test raised the same KeyError seen in the issue before this fix. Prior to this fix pandas used utf-8 as the default encoding.

I considered added an encoding kwarg to describe to let users control how they decode, but decided not to do that yet. Using utf-8-sig should support all the same tables that were supported by utf-8, so I figure until we hear demand for supporting other encodings we don't need to add this argument.

I also edited HISTORY.rst for an upcoming release. There are no other issues to complete before this release, and this saves an extra PR.

@davemfish
davemfish requested a review from phargogh September 4, 2026 13:32

@phargogh phargogh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks @davemfish !

@phargogh
phargogh merged commit aa769b3 into natcap:main Sep 4, 2026
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pyproject.toml requires python>=3.9, but 3.10 actually required '\ufeff' BOM issue when using geometamaker.describe() on a CSV

2 participants