Soft deprecate `re.match` and `re.Pattern.match` in favour of `prefixmatch`

[hugovk]

Feature or enhancement

Proposal:

In #86519 (PR #31137), we added new re.prefixmatch and re.Pattern.prefixmatch as explicit names for re.match and re.Pattern.match, because it's really not obvious the shorter old names include a prefix anchor; you probably meant to use search (no anchors) or fullmatch (prefix and suffix anchors).

This is essentially a soft deprecation of the match names:

• It's okay to use the old name in old code, but prefer the new name in new code.
• We don't plan on removing the old name or emitting deprecation warnings.

Let's make this explicit.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-148100: Soft deprecate re.match and re.Pattern.match in favour of prefixmatch #148101

[serhiy-storchaka]

My objection to prefixmath was exactly this -- that it will lead to deprecation of match. Even if it is initially a soft-deprecation, it will cause a tsunami of PRs in Python and other projects to replace match with prefixmath. This in unnecessary code churn that will affect each Python project.

I was told that this would never happen. And here it is.

[hugovk]

This is absolutely and explicitly not a regular "hard" deprecation.

A soft deprecation is clearly defined:

• Something which you should not use in new code.
• Safe for existing code.
• No deprecation warnings.

It's only in docs, which clearly says we do not plan to remove it.

#31137 was a soft deprecation; this is labelling it as such.

I completely agree that match too widespread to "hard" deprecate.

I don't think there'll be a tsunami of PRs, but I hope people take another look at their uses of match and consider if they should be search or fullmatch instead.

The first codebase I just searched has a match that should really be fullmatch. And for 3.15+ projects (many won't be for another five years), I do think explicit prefixmatch is clearer than match. But labelling this a soft deprecation doesn't change any of this.

[serhiy-storchaka]

How do you stop other people from creating PRs to replace match with prefixmatch? This is a low hanging fruit, we will get such PRs every few months, and finally some of them will slip past the watchdog. How do you stop this in other projects, not only in open source projects on GitHub? How do you stop linters to add a rule to recommend prefixmatch over match? And now it is easy to create LLM generated PRs to hundreds of projects.

Do you think all those man-hours spent globally reviewing and discussing such changes are worth it? On a global scale, it could be man-years.

[gpshead]

this is not a deprecation. this is a soft deprecation. match will never be removed.

i'm not worried about people creating useless PRs making unwanted changes, they already do that. this doesn't cause it. that's a social problem. we shouldn't hold the language back and refuse to make changes that make things more clear because we're worried someone might try and use them before the world is ready.

[serhiy-storchaka]

Soft deprecation will be seen as a first step of deprecation. There will be linter rules to enforce this, and there will be numerous PRs to "make things more clear". And since re.match() is used virtually in any project, such changes will have giant impact.

I cannot resist if several core developers support such change, but to me this is a large waste of our and other's resources.

[ambv]

Soft deprecation will be seen as a first step of deprecation.

Why do you say this? We literally have docs around soft deprecation and they explicitly state there's no causal connection between soft and hard deprecations. Do we have evidence of people confused by soft deprecations in the past?

[serhiy-storchaka]

Because this is just in human nature. Most people do not follow link, they see "soft deprecated", and read it as ligher deprecation. They remember that many other softdeprecated features are actively discouraged. And there is no promise that softdeprecated functions will not be deprecated.

And frankly, some of currently softdeprecated functions will be deprecated and finally removed, because they have inner flaws. It was just impractical to deprecate them before providing an alternative.

• Calling mimetypes.guess_type() with a path (instead of URL) argument. This is just ambiguous. New function mimetypes.guess_file_type() was added for paths. We can start hard deprecation in the next version.
• PyModule_AddObject() requires unusual handling to avoid reference leaks. Since its behavior different from behavior of any other C API function, and errors are rare, most code in the wild use it incorrectly. But it is so ubiquitous, that replacing it with better alternatives will cause large disturbance. We can start emitting runtime warnings only in distant future.

Maybe I used soft deprecation improperly, and it should be just long documentation-only term deprecation. But it added to impression that soft deprecation is a step to normal deprecation.

We cannot actually stop people from creating PRs to mass replace match() with prefixmatch(). But we can try to make this transition more gradual, when the new code uses prefixmatch() (it can only be used if the code drops support of 3.14), and the old code dies for natural causes. This is many years if not tens of years in future. We should not provoke mass rewriting. I am against the use of term "soft deprecated" and the deprecated directive (which is inappropriate for prefixmatch() in any case). Just ensure that prefixmatch() is mentioned first, that match() is somewhere mentioned as an alternative name, that index entry for match() and references lead to prefixmatch(), that it is documented that prefixmatch() was added in 3.15. Do not use word "deprecated" at all.

[hugovk]

Maybe I used soft deprecation improperly, and it should be just long documentation-only term deprecation. But it added to impression that soft deprecation is a step to normal deprecation.

Regular deprecation and removal could follow soft deprecation, but soft deprecation is absolutely not a step on the path to normal deprecation.

And there is no promise that softdeprecated functions will not be deprecated.

Indeed, we don't know what will happen in the future. But there's a promise that soft-deprecation -> deprecation is not the same process. They are separate things.

If someone wanted to fully deprecate re.match, we'd check the usage and likely very quickly dismiss it as not possible.

I am against the use of term "soft deprecated" and the deprecated directive (which is inappropriate for prefixmatch() in any case).

I'll be opening a followup PR to add a Sphinx soft-deprecated directive for these, that will also link to the definition, and we can use this instead of deprecated.

Do not use word "deprecated" at all.

It might be worth opening a Discuss topic to see if we can come up with another term for "soft-deprecation" that is less prone to confusion?

[serhiy-storchaka]

I'll be opening a followup PR to add a Sphinx soft-deprecated directive for these

This would also be incorrect. This is because there is a single entry shared between prefixmatch() and match(), so any directive added here will be applied to prefixmatch(), which would be incorrect.

[mpkocher]

Perhaps prefixmatch has the potential to improve visibility of the differences between match and search.

However, this alias approach will certainly be increasing entropy. This looks like a mixed bag at best and it's hard to see how creating an alias is going to shift the inertia of a 20+ year old naming mistake.

This entropy is also in other places and looks similar to re.error re.PatternError discussion. The soft depreciation mechanism seems more appropriate for re.error than for re.match.

#83162

Nevertheless, this entropy generates challenges when a PR in your team is using match. Should I correct them to use prefixmatch so that it's explicit? Or is it a seasoned ol' timer in my team who knows the difference between match/search and it's not an issue. Should we create linting rules to try to keep our code base consistent? Is Pycharm doing the same thing as VS Code? etc...

A different example would be urlsplit vs urlparse. People seem to be still using this incorrectly. Without hard depreciation, the inertia incorrect use is difficult to correct.

I wish there was a better path forward to handle these cases. This one obvious way ethos is having trouble with the strict backwards compatible and "stability" requirements of Python.