gh-128185: Align the grammar of the Decimal string constructor with float's

[HarryLHW]

This PR

• removes the mention of underscores in the text
• changes digits from digit [digit]... to (digit | "_")* digit (digit | "_")*
• makes it a .. productionlist::

• Issue: Align the grammar of the Decimal string constructor with float's #128185

---

📚 Documentation preview 📚: https://cpython-previews--128315.org.readthedocs.build/en/128315/library/decimal.html#decimal.Decimal

[picnixz]

I actually tested something and saw that Decimal("NaN__") is accepted as being a NaN. So, the grammar of NaN should also be changed. Initially, I thought that the grammar would be improved, but it might be easier to simply keep the mention in the text =/ (sorry @skirpichev, but you were probably right).

However, we can definitely use a production list instead.

[picnixz]

Do Sphinx production lists support numeric-value instead or should it be without any weird symbol? if not, use _ instead of - to separate words.

[HarryLHW]

Sphinx does not support -, but it does support _.

[picnixz]

Yet another thing I should perhaps add to my list of things to fix upstream... Thanks for checking.

[HarryLHW]

I actually tested something and saw that Decimal("NaN__") is accepted as being a NaN. So, the grammar of NaN should also be changed. Initially, I thought that the grammar would be improved, but it might be easier to simply keep the mention in the text =/ (sorry @skirpichev, but you were probably right).

However, we can definitely use a production list instead.

The implementation is

cpython/Modules/_decimal/_decimal.c

Lines 2135 to 2139 in 2cf396c

	for (; j < len; j++) {
	ch = PyUnicode_READ(kind, data, j);
	if (ignore_underscores && ch == '_') {
	continue;
	}

If I am correct, this ignores every '_' in the string no matter where it is, similar to string.replace('_', ''). Decimal('__I__N__F__') and Decimal('__N__A__N__') are also valid. It is difficult to generalize an expression for NaN or Infinity.

EDIT:
It is not difficult. The expression could be "_"* "N" "_"* "a" "_"* "N" "_"*, but very confusing.

[picnixz]

If I am correct, this ignores every '' in the string no matter where it is, similar to string.replace('', ''). Decimal('I__N__F') and Decimal('N__A__N') are also valid. It is difficult to generalize an expression for NaN or Infinity.

Considering this, I think we should keep the previous text. Sorry for this blunder but I didn't know that it also applied to NaN. However, we can use a production list.

[HarryLHW]

If I am correct, this ignores every '' in the string no matter where it is, similar to string.replace('', ''). Decimal('I__N__F') and Decimal('N__A__N') are also valid. It is difficult to generalize an expression for NaN or Infinity.

Considering this, I think we should keep the previous text. Sorry for this blunder but I didn't know that it also applied to NaN. However, we can use a production list.

OK. Should I keep the underscores in digits?
digits ::= (digit | "_")* digit (digit | "_")*

[picnixz]

OK. Should I keep the underscores in digits?

No, let's just revert the changes. It would be more confusing if we keep it in the grammar (especially considering how the pattern would actually read). Just transform the current grammar into a production list. However, before that, I'll ask some docs expert for their opinion @ncoghlan @willingc (the problem with me is that I wouldn't mind the change but we're already saying "we trim whitespaces" so we're already "pre-processing" the input to ease the grammar specifications).

[skirpichev]

As it was already noted in the issue thread, I don't think there is a bug.

New docs are inaccurate, as already noted by Benedict. While it's possible to "fix" this, I doubt that will be an improvement: new grammar will be too complex, perhaps misleading and reveal implementation details, which should be rather hidden.

[picnixz]

To summarize (unless our docs experts advise differently):

• let's not change the grammar or the text
• but let's change the grammar markup into a production list markup

I would recommend merging #128323 first and then this one or do both in one PR.

[skirpichev]

Looks good, except for few grammar fixes.

Though, maybe we want instead reference terms from other grammars, like https://docs.python.org/3/reference/lexical_analysis.html#floating-point-literals does (almost everything is same as for grammar in the float() docs: https://docs.python.org/3/library/functions.html#float).

But lets see what @picnixz think on this first. Maybe for readability it's better to repeat all in decimal docs.

[skirpichev]

Here you can reference grammar term instead. Though, this may add a conflict with #128323, so - you can include that change here as well.

[skirpichev]

Like this:

   Other Unicode decimal digits are also permitted where :token:`~decimal.digit`

[picnixz]

Maybe for readability it's better to repeat all in decimal docs.

I don't have a strong preference. For instance, I personally prefer digit [digit]... instead of (digit)+ but it's a matter of taste. Now, since we're far from the page about floats, maybe it's better to keep the verbose 0 | 1 | ... etc?

[skirpichev]

since we're far from the page about floats, maybe it's better to keep the verbose

Yes that were my thoughts as well. I think it's ok to repeat grammar terms, even if they introduced before on other pages.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.